[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:06 dist-test-slave-6zbq krb5kdc[17697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:06 dist-test-slave-6zbq krb5kdc[17697](info): set up 2 sockets
Aug 12 06:51:06 dist-test-slave-6zbq krb5kdc[17697](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 06:51:07.720643 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.411s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:07 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:07.892405 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:43991
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:08.234822 17719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:08.235180 17719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:08.235308 17719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:08.242195 17719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:08.242298 17719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:08.242390 17719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:08.242465 17719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:08.242523 17719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:08.252063 17719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:08.253660 17719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:08.255507 17719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:08.278309 17731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:08.278334 17734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:08.278223 17732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:08.279237 17719 server_base.cc:1034] running on GCE node
I20260812 06:51:08.280267 17719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:08.282292 17719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:08.283546 17719 hybrid_clock.cc:648] HybridClock initialized: now 1786517468283462 us; error 68 us; skew 500 ppm
I20260812 06:51:08.284086 17719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:08 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517468, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:08.287511 17719 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:08.289865 17719 webserver.cc:460] Webserver started at http://127.17.69.126:40413/ using document root <none> and password file <none>
I20260812 06:51:08.290933 17719 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:08.291062 17719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:08.291527 17719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:08.295849 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "2f4cd93a3d534617ae70223d60412397"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "2280f2548e90170cdc75cd77b71e6efd"
server_key_iv: "88c3e9d20fcf998c607be0ba7dc28a21"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.296989 17719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "2f4cd93a3d534617ae70223d60412397"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "2280f2548e90170cdc75cd77b71e6efd"
server_key_iv: "88c3e9d20fcf998c607be0ba7dc28a21"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.304738 17719 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260812 06:51:08.310223 17744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:08.312096 17719 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260812 06:51:08.312362 17719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "2f4cd93a3d534617ae70223d60412397"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "2280f2548e90170cdc75cd77b71e6efd"
server_key_iv: "88c3e9d20fcf998c607be0ba7dc28a21"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.312563 17719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:08.332918 17719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:08.337570 17719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:08.337881 17719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:08.358356 17719 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:43991
I20260812 06:51:08.358386 17829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:43991 every 8 connection(s)
I20260812 06:51:08.360482 17719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:08.367138 17830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:08.367255 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17719
I20260812 06:51:08.367616 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:08.369752 17685 external_mini_cluster.cc:1428] Setting key 08aad87ea4ba3d26f65fe75d9d3444d7
I20260812 06:51:08.383215 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397: Bootstrap starting.
I20260812 06:51:08.388923 17830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:08.390511 17830 log.cc:824] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:08.394416 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397: No bootstrap required, opened a new log
Aug 12 06:51:08 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:08.403410 17830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } }
I20260812 06:51:08.403915 17830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:08.404124 17830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4cd93a3d534617ae70223d60412397, State: Initialized, Role: FOLLOWER
I20260812 06:51:08.405230 17830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } }
I20260812 06:51:08.405550 17830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:08.405815 17830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:08.406018 17830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:08.408336 17830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } }
I20260812 06:51:08.409377 17830 leader_election.cc:302] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f4cd93a3d534617ae70223d60412397; no voters:
I20260812 06:51:08.410215 17830 leader_election.cc:288] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:08.410442 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:08.375639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55296 (local address 127.17.69.126:43991)
0812 06:51:08.376733 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:08.376751 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:08.378019 (+ 1268us) server_negotiation.cc:407] Connection header received
0812 06:51:08.387215 (+ 9196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:08.387269 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:08.387762 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:08.388165 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:08.391336 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.393054 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.394902 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.395681 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.395953 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:08.401389 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:08.401455 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:08.401489 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:08.401524 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:08.404706 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:08.405438 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:08.405446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:08.405454 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:08.405663 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:08.406082 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:08.406088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:08.406092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:08.406876 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:08.407348 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:08.408261 (+ 913us) server_negotiation.cc:299] Negotiation successful
0812 06:51:08.408652 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":384,"threads_started":1}
I20260812 06:51:08.410719 17840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:08.412590 17840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [term 1 LEADER]: Becoming Leader. State: Replica: 2f4cd93a3d534617ae70223d60412397, State: Running, Role: LEADER
I20260812 06:51:08.413340 17840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } }
I20260812 06:51:08.413573 17830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:08.418979 17842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f4cd93a3d534617ae70223d60412397. Latest consensus state: current_term: 1 leader_uuid: "2f4cd93a3d534617ae70223d60412397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } } }
I20260812 06:51:08.419013 17841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f4cd93a3d534617ae70223d60412397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cd93a3d534617ae70223d60412397" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 43991 } } }
I20260812 06:51:08.419430 17842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:08.419497 17841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:08.422454 17858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:08.430166 17858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:08.442756 17858 catalog_manager.cc:1347] Generated new cluster ID: 5da848b2fd624e518201ad51b63bc7e7
I20260812 06:51:08.442860 17858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:08.459203 17858 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:08.460537 17858 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:08.472651 17858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f4cd93a3d534617ae70223d60412397: Generated new TSK 0
I20260812 06:51:08.473500 17858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:08.520856 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:08.834802 17880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:08.835043 17880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:08.835160 17880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:08.841944 17880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:08.842036 17880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:08.842162 17880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:08.851362 17880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:08.852382 17880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:08.853546 17880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:08.866035 17890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:08.866063 17888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:08.866297 17892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:08.866979 17880 server_base.cc:1034] running on GCE node
I20260812 06:51:08.867537 17880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:08.868306 17880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:08.869493 17880 hybrid_clock.cc:648] HybridClock initialized: now 1786517468869460 us; error 27 us; skew 500 ppm
I20260812 06:51:08.869762 17880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:08 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517468, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:08.872728 17880 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:08.874034 17880 webserver.cc:460] Webserver started at http://127.17.69.65:44957/ using document root <none> and password file <none>
I20260812 06:51:08.874682 17880 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:08.874799 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:08.875067 17880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:08.877667 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "4df99c0f976e496daefe650a638ca2fe"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "b3c398cd9761b27725080d6c5f19e6ec"
server_key_iv: "08386dc7deac216bb902f1650cc1c7ea"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.878398 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "4df99c0f976e496daefe650a638ca2fe"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "b3c398cd9761b27725080d6c5f19e6ec"
server_key_iv: "08386dc7deac216bb902f1650cc1c7ea"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.882990 17880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 06:51:08.885982 17907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:08.887158 17880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:08.887298 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "4df99c0f976e496daefe650a638ca2fe"
format_stamp: "Formatted at 2026-08-12 06:51:08 on dist-test-slave-6zbq"
server_key: "b3c398cd9761b27725080d6c5f19e6ec"
server_key_iv: "08386dc7deac216bb902f1650cc1c7ea"
server_key_version: "encryptionkey@0"
I20260812 06:51:08.887462 17880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:08.925696 17880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:08.929291 17880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:08.929545 17880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:08.930564 17880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:08.932107 17880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:08.932183 17880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:08.932382 17880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:08.932446 17880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:08.964010 17880 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:42895
I20260812 06:51:08.964067 18118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:42895 every 8 connection(s)
I20260812 06:51:08.965140 17880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:08.965725 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17880
I20260812 06:51:08.965987 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:08.966810 17685 external_mini_cluster.cc:1428] Setting key 99e9b2e7bd4b985d0f2227467533ccc6
Aug 12 06:51:08 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517468, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
I20260812 06:51:08.986819 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:08.968413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:32913 (local address 127.17.69.126:43991)
0812 06:51:08.968837 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:08.968841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:08.969914 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 06:51:08.972306 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:08.972312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:08.972404 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:08.972500 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:08.974301 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.975165 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.976889 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.977483 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.977602 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:08.980753 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:08.980759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:08.980763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:08.980800 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:08.983885 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:08.984537 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:08.984542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:08.984546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:08.984700 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:08.985058 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:08.985064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:08.985067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:08.985431 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:08.985594 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:08.986213 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 06:51:08.986460 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":224,"threads_started":1}
I20260812 06:51:08.987442 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:08.968956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:43991 (local address 127.17.69.65:32913)
0812 06:51:08.969739 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:08.969788 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:08.971988 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:08.972752 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:08.972762 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:08.973429 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:08.974132 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:08.974154 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.975315 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.975320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:08.976751 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:08.976764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:08.977654 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:08.977659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:08.977853 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:08.978453 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:08.978473 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:08.980550 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:08.984012 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:08.984018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:08.984029 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:08.984427 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:08.984794 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:08.984797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:08.984799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:08.984958 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:08.985542 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:08.985547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:08.985802 (+ 255us) client_negotiation.cc:769] Sending connection context
0812 06:51:08.986065 (+ 263us) client_negotiation.cc:240] Negotiation successful
0812 06:51:08.986445 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":286,"threads_started":1}
I20260812 06:51:08.988930 18119 heartbeater.cc:347] Connected to a master server at 127.17.69.126:43991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
I20260812 06:51:08.989326 18119 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:08.990159 18119 heartbeater.cc:511] Master 127.17.69.126:43991 requested a full tablet report, sending...
I20260812 06:51:08.993834 17773 ts_manager.cc:194] Registered new tserver with Master: 4df99c0f976e496daefe650a638ca2fe (127.17.69.65:42895)
I20260812 06:51:08.996670 17773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:32913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:09.009972 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:09.319020 18132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:09.319254 18132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:09.319373 18132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:09.326237 18132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:09.326325 18132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:09.326498 18132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:09.335547 18132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:09.336601 18132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:09.337814 18132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:09.350047 18142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:09.350018 18149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:09.350083 18147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:09.350821 18132 server_base.cc:1034] running on GCE node
I20260812 06:51:09.351351 18132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:09.352119 18132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:09.353309 18132 hybrid_clock.cc:648] HybridClock initialized: now 1786517469353264 us; error 30 us; skew 500 ppm
I20260812 06:51:09.353580 18132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:09.356446 18132 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:09.357774 18132 webserver.cc:460] Webserver started at http://127.17.69.66:33725/ using document root <none> and password file <none>
I20260812 06:51:09.358614 18132 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:09.358745 18132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:09.359050 18132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:09.362727 18132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "c50a81bb784f4e24bf95bbca44bc68b3"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "411e8d4f4ad68ef483a5fd32af13e64c"
server_key_iv: "1c2275663559828e912e6276296b1d6c"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.363850 18132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "c50a81bb784f4e24bf95bbca44bc68b3"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "411e8d4f4ad68ef483a5fd32af13e64c"
server_key_iv: "1c2275663559828e912e6276296b1d6c"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.369930 18132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:51:09.373919 18160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.375340 18132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:51:09.375496 18132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "c50a81bb784f4e24bf95bbca44bc68b3"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "411e8d4f4ad68ef483a5fd32af13e64c"
server_key_iv: "1c2275663559828e912e6276296b1d6c"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.375694 18132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:09.402417 18132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:09.406025 18132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:09.406253 18132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:09.407105 18132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:09.408281 18132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:09.408350 18132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.408437 18132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:09.408479 18132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.439352 18132 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:36905
I20260812 06:51:09.439414 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:36905 every 8 connection(s)
I20260812 06:51:09.440572 18132 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:09.444676 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18132
I20260812 06:51:09.444855 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:09.445259 17685 external_mini_cluster.cc:1428] Setting key 6b34a76560fca4dea98fd7188539cc66
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:09.459941 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:09.442864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:34565 (local address 127.17.69.126:43991)
0812 06:51:09.443040 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:09.443044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:09.444126 (+ 1082us) server_negotiation.cc:407] Connection header received
0812 06:51:09.445997 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:09.446002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:09.446088 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:09.446196 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:09.447911 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.448900 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.450331 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.451005 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.451126 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:09.454303 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:09.454309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:09.454313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:09.454376 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:09.457073 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.457852 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.457855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.457858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.457979 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.458427 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.458431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.458433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.458690 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:09.458804 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:09.459524 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 06:51:09.459694 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:51:09.461117 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:09.443304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:43991 (local address 127.17.69.66:34565)
0812 06:51:09.443973 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:09.444017 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:09.445685 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:09.446446 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:09.446455 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:09.447131 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:09.447742 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:09.447759 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.449022 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.449025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:09.450179 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:09.450191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.451188 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.451193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:09.451382 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:09.451983 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:09.452003 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:09.454091 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:09.457217 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:09.457226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:09.457239 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:09.457760 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:09.458067 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:09.458072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:09.458075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:09.458299 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:09.458814 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:09.458821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:09.459164 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 06:51:09.459459 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 06:51:09.459910 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":259,"threads_started":1}
I20260812 06:51:09.462524 18346 heartbeater.cc:347] Connected to a master server at 127.17.69.126:43991
I20260812 06:51:09.462852 18346 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:09.463495 18346 heartbeater.cc:511] Master 127.17.69.126:43991 requested a full tablet report, sending...
I20260812 06:51:09.464968 17773 ts_manager.cc:194] Registered new tserver with Master: c50a81bb784f4e24bf95bbca44bc68b3 (127.17.69.66:36905)
I20260812 06:51:09.465744 17773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:34565
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:09.491401 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:09.803030 18356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:09.803306 18356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:09.803448 18356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:09.810276 18356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:09.810403 18356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:09.810554 18356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:09.819823 18356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:43991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:09.820926 18356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:09.822152 18356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:09.834230 18368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:09.834295 18370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:09.834446 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:09.834967 18356 server_base.cc:1034] running on GCE node
I20260812 06:51:09.835613 18356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:09.836417 18356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:09.837615 18356 hybrid_clock.cc:648] HybridClock initialized: now 1786517469837556 us; error 39 us; skew 500 ppm
I20260812 06:51:09.837896 18356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:09.840721 18356 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:09.841981 18356 webserver.cc:460] Webserver started at http://127.17.69.67:39967/ using document root <none> and password file <none>
I20260812 06:51:09.842623 18356 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:09.842739 18356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:09.843024 18356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:09.845834 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "52a9bda0629e4a58b4093e4b0d863cea"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "fe158b3b4299d6269d5cad64a60d702b"
server_key_iv: "5a02a9b5011da41878d9ff837ef954d1"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.846556 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "52a9bda0629e4a58b4093e4b0d863cea"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "fe158b3b4299d6269d5cad64a60d702b"
server_key_iv: "5a02a9b5011da41878d9ff837ef954d1"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.851380 18356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:51:09.854564 18383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.855693 18356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:09.855837 18356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "52a9bda0629e4a58b4093e4b0d863cea"
format_stamp: "Formatted at 2026-08-12 06:51:09 on dist-test-slave-6zbq"
server_key: "fe158b3b4299d6269d5cad64a60d702b"
server_key_iv: "5a02a9b5011da41878d9ff837ef954d1"
server_key_version: "encryptionkey@0"
I20260812 06:51:09.856002 18356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:09.875722 18356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:09.879124 18356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:09.879362 18356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:09.880215 18356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:09.881426 18356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:09.881503 18356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.881624 18356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:09.881678 18356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:09.918213 18356 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:37257
I20260812 06:51:09.918272 18583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:37257 every 8 connection(s)
I20260812 06:51:09.919517 18356 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:09.927839 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18356
I20260812 06:51:09.928006 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:09.928427 17685 external_mini_cluster.cc:1428] Setting key d43fa11168b3fc0cb776874e8c275a01
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:09.937683 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:09.921930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:40025 (local address 127.17.69.126:43991)
0812 06:51:09.922100 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:09.922104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:09.923324 (+ 1220us) server_negotiation.cc:407] Connection header received
0812 06:51:09.924993 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:09.924998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:09.925065 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:09.925155 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:09.926496 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.927223 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.928716 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.929126 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.929255 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:09.933046 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:09.933051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:09.933054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:09.933085 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:09.935346 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.935941 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.935944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.935946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.936052 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.936379 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.936382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.936384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.936633 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:09.936738 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:09.937268 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 06:51:09.937434 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:51:09.938544 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:09.922454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:43991 (local address 127.17.69.67:40025)
0812 06:51:09.923139 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:09.923175 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:09.924772 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:09.925330 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:09.925337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:09.925824 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:09.926332 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:09.926374 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.927330 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.927334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:09.928580 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:09.928593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.929330 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.929335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:09.929617 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:09.930286 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:09.930322 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:09.932845 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:09.935455 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:09.935462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:09.935472 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:09.935853 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:09.936125 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:09.936128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:09.936131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:09.936300 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:09.936718 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:09.936723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:09.936981 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 06:51:09.937196 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 06:51:09.937518 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":265,"threads_started":1}
I20260812 06:51:09.939683 18586 heartbeater.cc:347] Connected to a master server at 127.17.69.126:43991
I20260812 06:51:09.939918 18586 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:09.940511 18586 heartbeater.cc:511] Master 127.17.69.126:43991 requested a full tablet report, sending...
I20260812 06:51:09.941747 17773 ts_manager.cc:194] Registered new tserver with Master: 52a9bda0629e4a58b4093e4b0d863cea (127.17.69.67:37257)
I20260812 06:51:09.942591 17773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:40025
I20260812 06:51:09.944872 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:51:09 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:09.994306 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:09.979601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.17.69.65:42895)
0812 06:51:09.980157 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:09.980164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:09.980428 (+ 264us) server_negotiation.cc:407] Connection header received
0812 06:51:09.980653 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:09.980661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:09.980882 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:09.981208 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:09.982595 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.983624 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.984792 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:09.985306 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:09.985402 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:09.988989 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:09.988999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:09.989011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:09.989047 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:09.991937 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.992502 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.992508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.992510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.992638 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:09.992990 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:09.992994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:09.992997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:09.993378 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:09.993616 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:09.993812 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 06:51:09.994004 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
W20260812 06:51:09.995550 17963 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32852
I20260812 06:51:10.000161 18119 heartbeater.cc:503] Master 127.17.69.126:43991 was elected leader, sending a full tablet report...
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[17697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:10.042603 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:10.029550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.17.69.65:42895)
0812 06:51:10.029769 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:10.029776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:10.029963 (+ 187us) server_negotiation.cc:407] Connection header received
0812 06:51:10.030172 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:10.030176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:10.030254 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:10.030369 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:10.031675 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.032374 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.033580 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.033982 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.034092 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:10.037773 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:10.037778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:10.037781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:10.037813 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:10.040186 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.040770 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.040775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.040777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.040899 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.041277 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.041282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.041285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.041533 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:10.041770 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:10.042060 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 06:51:10.042275 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:51:10.044507 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17880
I20260812 06:51:10.056443 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:10.045551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.17.69.65:42895)
0812 06:51:10.045746 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:10.045751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:10.045907 (+ 156us) server_negotiation.cc:407] Connection header received
0812 06:51:10.046013 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:10.046017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:10.046070 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:10.046145 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:10.047234 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.047892 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.049022 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.049430 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.049518 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:10.051606 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:10.051611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:10.051614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:10.051642 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:10.053765 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.054323 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.054326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.054329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.054482 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.054850 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.054854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.054857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.055199 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:10.055426 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:10.055778 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 06:51:10.055993 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:51:10.057355 18110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:10.145231 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17880
I20260812 06:51:10.166953 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18132
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:10.182058 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:10.167860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.17.69.66:36905)
0812 06:51:10.168420 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:10.168427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:10.168454 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:51:10.168519 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:10.168525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:10.168743 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:10.168867 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:10.170045 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.171167 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.172343 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.172875 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.172969 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:10.176729 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:10.176740 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:10.176754 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:10.176787 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:10.179584 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.180128 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.180135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.180137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.180256 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.180582 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.180586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.180589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.180969 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:10.181220 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:10.181488 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 06:51:10.181701 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":288,"threads_started":1}
I20260812 06:51:10.183368 18339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:10.263211 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18132
I20260812 06:51:10.284031 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18356
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[17697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:10.298894 18613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:10.284947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.17.69.67:37257)
0812 06:51:10.285542 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:10.285550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:10.285579 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:51:10.285641 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:10.285648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:10.285864 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:10.285983 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:10.287157 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.288157 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.289381 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:10.289908 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:10.289999 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:10.293818 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:10.293829 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:10.293841 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:10.293874 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:10.296536 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.297064 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.297070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.297072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.297184 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:10.297541 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:10.297544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:10.297546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:10.297861 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:10.298098 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:10.298306 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:51:10.298530 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":278,"threads_started":1}
I20260812 06:51:10.300149 18577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:10.381654 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18356
I20260812 06:51:10.404456 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17719
I20260812 06:51:10.405678 17825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:10.508095 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17719
2026-08-12T06:51:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4260 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[18628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[18628](info): set up 2 sockets
Aug 12 06:51:10 dist-test-slave-6zbq krb5kdc[18628](info): commencing operation
krb5kdc: starting...
W20260812 06:51:11.733174 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:11 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:11.909024 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36145
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:12.217473 18648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:12.217732 18648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:12.217866 18648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:12.224759 18648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:12.224858 18648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:12.224934 18648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:12.225011 18648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:12.225070 18648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:12.234155 18648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:12.235231 18648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:12.236389 18648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:51:12.248706 18648 server_base.cc:1034] running on GCE node
W20260812 06:51:12.248664 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:12.248641 18663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:12.249087 18660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:12.250604 18648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:12.252610 18648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:12.253882 18648 hybrid_clock.cc:648] HybridClock initialized: now 1786517472253824 us; error 55 us; skew 500 ppm
I20260812 06:51:12.254300 18648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:12 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:12.260689 18648 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:12.262379 18648 webserver.cc:460] Webserver started at http://127.17.69.126:38147/ using document root <none> and password file <none>
I20260812 06:51:12.263095 18648 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:12.263218 18648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:12.263522 18648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:12.266569 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "58b9584e7ea10f41b79706120bb3d79a"
server_key_iv: "61116f6a99a142515c701fa2202e9a17"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.267324 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "58b9584e7ea10f41b79706120bb3d79a"
server_key_iv: "61116f6a99a142515c701fa2202e9a17"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.272439 18648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:51:12.275619 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:12.276809 18648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:12.276971 18648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "58b9584e7ea10f41b79706120bb3d79a"
server_key_iv: "61116f6a99a142515c701fa2202e9a17"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.277136 18648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:12.291880 18648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:12.295799 18648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:12.296046 18648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:12.313479 18648 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:36145
I20260812 06:51:12.313509 18761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:36145 every 8 connection(s)
I20260812 06:51:12.314796 18648 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:12.318321 18763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:12.323182 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18648
I20260812 06:51:12.323388 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:12.324029 17685 external_mini_cluster.cc:1428] Setting key 72937264548b256b9dbd2c382199fdb0
I20260812 06:51:12.329710 18763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4: Bootstrap starting.
I20260812 06:51:12.333276 18763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:12.334280 18763 log.cc:824] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:12.337142 18763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4: No bootstrap required, opened a new log
Aug 12 06:51:12 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:12.345955 18763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } }
I20260812 06:51:12.346432 18763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:12.346539 18763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a1d58f5f0e47f9a0c67ff37aec70d4, State: Initialized, Role: FOLLOWER
I20260812 06:51:12.347064 18763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } }
I20260812 06:51:12.347292 18763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:12.347424 18763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:12.347455 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:12.325900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.17.69.126:36145)
0812 06:51:12.326869 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:12.326884 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:12.326934 (+ 50us) server_negotiation.cc:407] Connection header received
0812 06:51:12.329234 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:12.329265 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:12.329728 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:12.330132 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:12.331511 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.333389 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.334615 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.335413 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.335587 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:12.340002 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:12.340010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:12.340027 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:12.340062 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:12.343314 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:12.344103 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:12.344110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:12.344116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:12.344288 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:12.344665 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:12.344668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:12.344670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:12.345213 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:12.345433 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:12.345949 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 06:51:12.346290 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":463,"threads_started":1}
I20260812 06:51:12.347599 18763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:12.350222 18763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } }
I20260812 06:51:12.350746 18763 leader_election.cc:302] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a1d58f5f0e47f9a0c67ff37aec70d4; no voters:
I20260812 06:51:12.351258 18763 leader_election.cc:288] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:12.351385 18773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:12.352011 18773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [term 1 LEADER]: Becoming Leader. State: Replica: 21a1d58f5f0e47f9a0c67ff37aec70d4, State: Running, Role: LEADER
I20260812 06:51:12.352533 18773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } }
I20260812 06:51:12.353130 18763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:12.355120 18774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } } }
I20260812 06:51:12.355520 18774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:12.355381 18778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a1d58f5f0e47f9a0c67ff37aec70d4. Latest consensus state: current_term: 1 leader_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a1d58f5f0e47f9a0c67ff37aec70d4" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36145 } } }
I20260812 06:51:12.355676 18778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:12.357487 18789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:12.362032 18789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:12.369684 18789 catalog_manager.cc:1347] Generated new cluster ID: 7bca53e5f7824d90be621db93ff040fd
I20260812 06:51:12.369781 18789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:12.386911 18789 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:12.388057 18789 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:12.396392 18789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a1d58f5f0e47f9a0c67ff37aec70d4: Generated new TSK 0
I20260812 06:51:12.397089 18789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:12.441443 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:12.760643 18811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:12.760915 18811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:12.761013 18811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:12.768603 18811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:12.768718 18811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:12.768836 18811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:12.778518 18811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:12.779654 18811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:12.780934 18811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:12.793391 18822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:12.793426 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:12.793467 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:12.794181 18811 server_base.cc:1034] running on GCE node
I20260812 06:51:12.794799 18811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:12.795653 18811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:12.796886 18811 hybrid_clock.cc:648] HybridClock initialized: now 1786517472796827 us; error 40 us; skew 500 ppm
I20260812 06:51:12.797302 18811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:12 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:12.800423 18811 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:12.801781 18811 webserver.cc:460] Webserver started at http://127.17.69.65:35195/ using document root <none> and password file <none>
I20260812 06:51:12.802464 18811 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:12.802567 18811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:12.802820 18811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:12.805661 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "62e127c1a09e4b0c8f757cff1578528c"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "4b4a5195074be7dcfe8938b034801233"
server_key_iv: "c4d548745a8dd640c6cd304081380cec"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.806512 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "62e127c1a09e4b0c8f757cff1578528c"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "4b4a5195074be7dcfe8938b034801233"
server_key_iv: "c4d548745a8dd640c6cd304081380cec"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.811844 18811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 06:51:12.815294 18836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:12.816744 18811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:51:12.816888 18811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "62e127c1a09e4b0c8f757cff1578528c"
format_stamp: "Formatted at 2026-08-12 06:51:12 on dist-test-slave-6zbq"
server_key: "4b4a5195074be7dcfe8938b034801233"
server_key_iv: "c4d548745a8dd640c6cd304081380cec"
server_key_version: "encryptionkey@0"
I20260812 06:51:12.817063 18811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:12.832775 18811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:12.836901 18811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:12.837134 18811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:12.838028 18811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:12.839356 18811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:12.839427 18811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:12.839526 18811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:12.839568 18811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:12.869359 18811 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:36611
I20260812 06:51:12.869401 19034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:36611 every 8 connection(s)
I20260812 06:51:12.870735 18811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:12.877503 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18811
I20260812 06:51:12.877694 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:12.878283 17685 external_mini_cluster.cc:1428] Setting key 61607bbf2d61cdf6d4a3129a1eaa3819
Aug 12 06:51:12 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517472, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:12.890983 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:12.873875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:59093 (local address 127.17.69.126:36145)
0812 06:51:12.874284 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:12.874289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:12.875172 (+ 883us) server_negotiation.cc:407] Connection header received
0812 06:51:12.876949 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:12.876955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:12.877036 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:12.877169 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:12.878764 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.879694 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.881256 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.881835 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.881971 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:12.885260 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:12.885266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:12.885269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:12.885299 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:12.888069 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:12.888860 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:12.888864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:12.888867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:12.888989 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:12.889426 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:12.889430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:12.889432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:12.889692 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:12.889802 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:12.890569 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 06:51:12.890731 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":207,"threads_started":1}
I20260812 06:51:12.892184 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:12.874296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36145 (local address 127.17.69.65:59093)
0812 06:51:12.875019 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:12.875055 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:12.876690 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:12.877401 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:12.877409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:12.877989 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:12.878581 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:12.878611 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.879847 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.879851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:12.881114 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:12.881131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:12.882084 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:12.882102 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:12.882379 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:12.882993 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:12.883015 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:12.885042 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:12.888197 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:12.888206 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:12.888221 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:12.888733 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:12.889082 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:12.889087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:12.889091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:12.889332 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:12.889810 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:12.889817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:12.890191 (+ 374us) client_negotiation.cc:769] Sending connection context
0812 06:51:12.890480 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 06:51:12.890978 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":264,"threads_started":1}
I20260812 06:51:12.893618 19035 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36145
I20260812 06:51:12.893927 19035 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:12.894840 19035 heartbeater.cc:511] Master 127.17.69.126:36145 requested a full tablet report, sending...
I20260812 06:51:12.896939 18703 ts_manager.cc:194] Registered new tserver with Master: 62e127c1a09e4b0c8f757cff1578528c (127.17.69.65:36611)
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
I20260812 06:51:12.898362 18703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:59093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:12.920485 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:13.228665 19049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:13.228906 19049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:13.229036 19049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:13.235926 19049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:13.236017 19049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:13.236156 19049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:13.245445 19049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:13.246595 19049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:13.247819 19049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:13.260011 19058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:13.260172 19059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:13.260346 19061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:13.261133 19049 server_base.cc:1034] running on GCE node
I20260812 06:51:13.261734 19049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:13.262665 19049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:13.263846 19049 hybrid_clock.cc:648] HybridClock initialized: now 1786517473263796 us; error 44 us; skew 500 ppm
I20260812 06:51:13.264117 19049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:13 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517473, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:13.267074 19049 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:13.268426 19049 webserver.cc:460] Webserver started at http://127.17.69.66:38187/ using document root <none> and password file <none>
I20260812 06:51:13.269047 19049 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:13.269155 19049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:13.269436 19049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:13.272197 19049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "e60b24344a0645d4a4a07961ca15533b"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "c9bebdb65c0ab529be6c6075cd4136f2"
server_key_iv: "7c17aa9ca7118416f7ad315813598502"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.272943 19049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "e60b24344a0645d4a4a07961ca15533b"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "c9bebdb65c0ab529be6c6075cd4136f2"
server_key_iv: "7c17aa9ca7118416f7ad315813598502"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.277732 19049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:51:13.281008 19069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.282236 19049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:13.282403 19049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "e60b24344a0645d4a4a07961ca15533b"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "c9bebdb65c0ab529be6c6075cd4136f2"
server_key_iv: "7c17aa9ca7118416f7ad315813598502"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.282585 19049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:13.299718 19049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:13.303601 19049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:13.303857 19049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:13.304741 19049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:13.306010 19049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:13.306087 19049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.306209 19049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:13.306263 19049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.336409 19049 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:33801
I20260812 06:51:13.336486 19272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:33801 every 8 connection(s)
I20260812 06:51:13.337581 19049 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:13.345199 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19049
I20260812 06:51:13.345376 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:13.345794 17685 external_mini_cluster.cc:1428] Setting key e394979c76209f0394464a5fe76b1cd8
Aug 12 06:51:13 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517473, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:13.357944 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.339988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:48735 (local address 127.17.69.126:36145)
0812 06:51:13.340185 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.340189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.341293 (+ 1104us) server_negotiation.cc:407] Connection header received
0812 06:51:13.343409 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.343416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.343496 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.343584 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:13.345118 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.345945 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.347573 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.347988 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.348100 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.351980 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:13.351985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:13.351988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:13.352020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:13.354891 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.355725 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.355729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.355731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.355871 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.356287 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.356292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.356296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.356618 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:13.356769 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.357402 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.357623 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:51:13.358778 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.340437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36145 (local address 127.17.69.66:48735)
0812 06:51:13.341137 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.341177 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.343060 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.343820 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.343827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.344358 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:13.344917 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.344942 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.346121 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.346127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.347427 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.347440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.348212 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.348219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.348430 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.349111 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:13.349145 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:13.351754 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:13.355026 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:13.355036 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:13.355053 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:13.355613 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:13.355976 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:13.355979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:13.355982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:13.356176 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:13.356719 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:13.356724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:13.357013 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.357247 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.357646 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":280,"threads_started":1}
I20260812 06:51:13.360255 19273 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36145
I20260812 06:51:13.360524 19273 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:13.361222 19273 heartbeater.cc:511] Master 127.17.69.126:36145 requested a full tablet report, sending...
I20260812 06:51:13.362684 18703 ts_manager.cc:194] Registered new tserver with Master: e60b24344a0645d4a4a07961ca15533b (127.17.69.66:33801)
I20260812 06:51:13.363847 18703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:48735
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:13.389127 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:13.704137 19280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:13.704406 19280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:13.704528 19280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:13.711901 19280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:13.712006 19280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:13.712153 19280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:13.721771 19280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36145
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:13.723040 19280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:13.724314 19280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:13.736109 19287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:13.736155 19292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:13.736268 19285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:13.737049 19280 server_base.cc:1034] running on GCE node
I20260812 06:51:13.737560 19280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:13.738379 19280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:13.739576 19280 hybrid_clock.cc:648] HybridClock initialized: now 1786517473739526 us; error 45 us; skew 500 ppm
I20260812 06:51:13.739854 19280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:13 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517473, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:13.742668 19280 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:13.743961 19280 webserver.cc:460] Webserver started at http://127.17.69.67:40315/ using document root <none> and password file <none>
I20260812 06:51:13.744565 19280 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:13.744675 19280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:13.744953 19280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:13.747699 19280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "e577d9c828224c7eb6b7c9686c61967e"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "893ddf65f4f8c48899ca791998ab91d6"
server_key_iv: "96248b8ebff75df41c48935d98c636db"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.748431 19280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "e577d9c828224c7eb6b7c9686c61967e"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "893ddf65f4f8c48899ca791998ab91d6"
server_key_iv: "96248b8ebff75df41c48935d98c636db"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.753108 19280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:51:13.756410 19299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.757627 19280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:13.757794 19280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "e577d9c828224c7eb6b7c9686c61967e"
format_stamp: "Formatted at 2026-08-12 06:51:13 on dist-test-slave-6zbq"
server_key: "893ddf65f4f8c48899ca791998ab91d6"
server_key_iv: "96248b8ebff75df41c48935d98c636db"
server_key_version: "encryptionkey@0"
I20260812 06:51:13.757999 19280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:13.786127 19280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:13.790162 19280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:13.790436 19280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:13.791566 19280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:13.792847 19280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:13.792917 19280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.793004 19280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:13.793046 19280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:13.823506 19280 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:40875
I20260812 06:51:13.823557 19502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:40875 every 8 connection(s)
I20260812 06:51:13.824658 19280 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:13.834597 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19280
I20260812 06:51:13.834815 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:13.835348 17685 external_mini_cluster.cc:1428] Setting key a317f54fded2eea2b3e05333b281bbfc
Aug 12 06:51:13 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517473, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:13.843369 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.826959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:40719 (local address 127.17.69.126:36145)
0812 06:51:13.827134 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.827138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.828207 (+ 1069us) server_negotiation.cc:407] Connection header received
0812 06:51:13.830013 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.830018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.830093 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.830190 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:13.831709 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.832475 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.833922 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.834319 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.834446 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.838283 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:13.838288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:13.838291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:13.838323 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:13.840584 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.841242 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.841248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.841252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.841373 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.841765 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.841770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.841774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.842025 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:13.842156 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.842823 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.843041 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:51:13.844209 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.827385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36145 (local address 127.17.69.67:40719)
0812 06:51:13.828040 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.828084 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.829739 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.830480 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.830488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.830993 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:13.831538 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.831554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.832588 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.832592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.833790 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.833806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.834557 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.834564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.834875 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.835507 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:13.835534 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:13.838082 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:13.840711 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:13.840719 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:13.840733 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:13.841131 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:13.841473 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:13.841478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:13.841482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:13.841656 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:13.842147 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:13.842154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:13.842488 (+ 334us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.842715 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.843181 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":250,"threads_started":1}
I20260812 06:51:13.845465 19504 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36145
I20260812 06:51:13.845737 19504 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:13.846421 19504 heartbeater.cc:511] Master 127.17.69.126:36145 requested a full tablet report, sending...
I20260812 06:51:13.847705 18703 ts_manager.cc:194] Registered new tserver with Master: e577d9c828224c7eb6b7c9686c61967e (127.17.69.67:40875)
I20260812 06:51:13.848461 18703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:40719
I20260812 06:51:13.850467 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:51:13.869063 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.857175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.17.69.126:36145)
0812 06:51:13.857311 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.857315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.857857 (+ 542us) server_negotiation.cc:407] Connection header received
0812 06:51:13.858018 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.858023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.858082 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.858178 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:13.859735 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.860427 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.861757 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.862202 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.862305 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.864425 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:13.864430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:13.864433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:13.864461 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:13.866693 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.867251 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.867255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.867258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.867373 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:13.867713 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:13.867718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:13.867721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:13.868007 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:13.868237 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.868552 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.868752 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:51:13.877576 18703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:13.885648 18703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:13.901662 19035 heartbeater.cc:503] Master 127.17.69.126:36145 was elected leader, sending a full tablet report...
I20260812 06:51:13.921650 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.902061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56152 (local address 127.17.69.66:33801)
0812 06:51:13.911609 (+ 9548us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.911619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.912797 (+ 1178us) server_negotiation.cc:407] Connection header received
0812 06:51:13.912922 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.912930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.913208 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.913389 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:13.913964 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.915518 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.918097 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.919572 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.919699 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.919891 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.921230 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.921393 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9395,"thread_start_us":9256,"threads_started":1}
I20260812 06:51:13.921684 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.902401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.17.69.65:36611)
0812 06:51:13.914067 (+ 11666us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.914076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.914101 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:51:13.914199 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.914206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.914439 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.914618 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:13.915313 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.916824 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.918947 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.920122 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.920867 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.921118 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.921200 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.921361 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11501,"thread_start_us":11324,"threads_started":1}
I20260812 06:51:13.922184 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.901679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:33801 (local address 127.0.0.1:56152)
0812 06:51:13.912549 (+ 10870us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.912564 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.912670 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.913413 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.913419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.913447 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:13.913769 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.913782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.915641 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.915646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.917907 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.917926 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.920222 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.920229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.920455 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.920512 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.920711 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.920905 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10745,"mutex_wait_us":4,"thread_start_us":10632,"threads_started":1}
I20260812 06:51:13.922272 19518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.900541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40875 (local address 127.0.0.1:54806)
0812 06:51:13.901304 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.901348 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.901559 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.914969 (+ 13410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.914974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.915001 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:13.915388 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.915398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.917228 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.917234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.919277 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.919293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.921118 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.921127 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.921314 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.921346 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.921407 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.921526 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":51,"thread_start_us":319,"threads_started":1}
I20260812 06:51:13.922297 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.901153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:36611 (local address 127.0.0.1:32830)
0812 06:51:13.911920 (+ 10767us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.911940 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.912064 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.914628 (+ 2564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.914633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.914659 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:13.915114 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.915130 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.916958 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.916963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.918809 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.918822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.920392 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.920399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.920561 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.920597 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.920740 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.920915 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10629,"mutex_wait_us":90,"thread_start_us":10477,"threads_started":1}
I20260812 06:51:13.923168 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.900774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.17.69.67:40875)
0812 06:51:13.914441 (+ 13667us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.914446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.914472 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:51:13.914543 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.914547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.914764 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.914960 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:13.915568 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.917075 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.919445 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.920868 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.921718 (+ 850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.922624 (+ 906us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.922710 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.922871 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13490,"thread_start_us":13340,"threads_started":1}
I20260812 06:51:13.926220 19156 tablet_service.cc:1467] Processing CreateTablet for tablet 49cbebe4141940a8ac0fa155584e0107 (DEFAULT_TABLE table=test-table [id=5a753db6cadf415db1ba7b4ab7882294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:13.927940 19156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cbebe4141940a8ac0fa155584e0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:13.927894 19381 tablet_service.cc:1467] Processing CreateTablet for tablet 49cbebe4141940a8ac0fa155584e0107 (DEFAULT_TABLE table=test-table [id=5a753db6cadf415db1ba7b4ab7882294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:13.927858 18915 tablet_service.cc:1467] Processing CreateTablet for tablet 49cbebe4141940a8ac0fa155584e0107 (DEFAULT_TABLE table=test-table [id=5a753db6cadf415db1ba7b4ab7882294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:13.928988 19381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cbebe4141940a8ac0fa155584e0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:13.929250 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cbebe4141940a8ac0fa155584e0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:13.940320 19533 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Bootstrap starting.
I20260812 06:51:13.942715 19535 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Bootstrap starting.
I20260812 06:51:13.944729 19533 tablet_bootstrap.cc:654] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:13.945983 19533 log.cc:824] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:13.946053 19535 tablet_bootstrap.cc:654] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:13.947281 19535 log.cc:824] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:13.947397 19534 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Bootstrap starting.
I20260812 06:51:13.948864 19533 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: No bootstrap required, opened a new log
I20260812 06:51:13.949108 19533 ts_tablet_manager.cc:1397] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 06:51:13.950939 19534 tablet_bootstrap.cc:654] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:13.952170 19534 log.cc:824] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:13.954973 19534 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: No bootstrap required, opened a new log
I20260812 06:51:13.955206 19534 ts_tablet_manager.cc:1397] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 06:51:13.955466 19535 tablet_bootstrap.cc:492] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: No bootstrap required, opened a new log
I20260812 06:51:13.955704 19535 ts_tablet_manager.cc:1397] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 06:51:13.958112 19533 raft_consensus.cc:348] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.958524 19533 raft_consensus.cc:374] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:13.958632 19533 raft_consensus.cc:729] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e577d9c828224c7eb6b7c9686c61967e, State: Initialized, Role: FOLLOWER
I20260812 06:51:13.959312 19533 consensus_queue.cc:260] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.960507 19504 heartbeater.cc:503] Master 127.17.69.126:36145 was elected leader, sending a full tablet report...
I20260812 06:51:13.960904 19533 ts_tablet_manager.cc:1428] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260812 06:51:13.963534 19534 raft_consensus.cc:348] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.963835 19534 raft_consensus.cc:374] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:13.963937 19534 raft_consensus.cc:729] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60b24344a0645d4a4a07961ca15533b, State: Initialized, Role: FOLLOWER
I20260812 06:51:13.964455 19534 consensus_queue.cc:260] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.965420 19273 heartbeater.cc:503] Master 127.17.69.126:36145 was elected leader, sending a full tablet report...
I20260812 06:51:13.966006 19534 ts_tablet_manager.cc:1428] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260812 06:51:13.967763 19535 raft_consensus.cc:348] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.968065 19535 raft_consensus.cc:374] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:13.968163 19535 raft_consensus.cc:729] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e127c1a09e4b0c8f757cff1578528c, State: Initialized, Role: FOLLOWER
I20260812 06:51:13.968719 19535 consensus_queue.cc:260] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.969810 19543 raft_consensus.cc:482] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:13.970067 19543 raft_consensus.cc:504] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.970407 19535 ts_tablet_manager.cc:1428] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260812 06:51:13.972121 19543 leader_election.cc:288] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e577d9c828224c7eb6b7c9686c61967e (127.17.69.67:40875), e60b24344a0645d4a4a07961ca15533b (127.17.69.66:33801)
I20260812 06:51:13.979702 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.972775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:48209 (local address 127.17.69.67:40875)
0812 06:51:13.972956 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.972962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.973122 (+ 160us) server_negotiation.cc:407] Connection header received
0812 06:51:13.973415 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.973420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.973492 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.973559 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:13.974380 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.975421 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.977988 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.978789 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.978907 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.978958 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.979324 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.979431 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:51:13.979844 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.972435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40875 (local address 127.17.69.65:48209)
0812 06:51:13.973026 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.973045 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.973227 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.973772 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.973776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.973801 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:13.974209 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.974222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.975574 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.975579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.977827 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.977844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.978986 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.978993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.979163 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.979199 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.979299 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.979471 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":307,"threads_started":1}
I20260812 06:51:13.980333 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.973216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:40183 (local address 127.17.69.66:33801)
0812 06:51:13.973389 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:13.973393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:13.973754 (+ 361us) server_negotiation.cc:407] Connection header received
0812 06:51:13.973993 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:13.973998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:13.974056 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:13.974135 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:13.975031 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.975998 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.978401 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.979359 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.979517 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.979584 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:13.979985 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 06:51:13.980111 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:51:13.980389 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:13.973107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:33801 (local address 127.17.69.65:40183)
0812 06:51:13.973662 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:13.973680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:13.973862 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:13.974170 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:13.974175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:13.974201 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:13.974843 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.974858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.976176 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.976185 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.978227 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:13.978238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:13.979600 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:13.979605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:13.979777 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:13.979810 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:51:13.979955 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:51:13.980081 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":271,"threads_started":1}
I20260812 06:51:13.980888 19417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cbebe4141940a8ac0fa155584e0107" candidate_uuid: "62e127c1a09e4b0c8f757cff1578528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e577d9c828224c7eb6b7c9686c61967e" is_pre_election: true
I20260812 06:51:13.981616 19417 raft_consensus.cc:2454] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e127c1a09e4b0c8f757cff1578528c in term 0.
I20260812 06:51:13.981940 19187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cbebe4141940a8ac0fa155584e0107" candidate_uuid: "62e127c1a09e4b0c8f757cff1578528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60b24344a0645d4a4a07961ca15533b" is_pre_election: true
I20260812 06:51:13.982306 19187 raft_consensus.cc:2454] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e127c1a09e4b0c8f757cff1578528c in term 0.
I20260812 06:51:13.982391 18844 leader_election.cc:302] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e127c1a09e4b0c8f757cff1578528c, e577d9c828224c7eb6b7c9686c61967e; no voters:
I20260812 06:51:13.982725 19543 raft_consensus.cc:2781] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:13.982867 19543 raft_consensus.cc:482] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:13.982977 19543 raft_consensus.cc:3037] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:13.984963 19543 raft_consensus.cc:504] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.985457 19543 leader_election.cc:288] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [CANDIDATE]: Term 1 election: Requested vote from peers e577d9c828224c7eb6b7c9686c61967e (127.17.69.67:40875), e60b24344a0645d4a4a07961ca15533b (127.17.69.66:33801)
I20260812 06:51:13.985836 19417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cbebe4141940a8ac0fa155584e0107" candidate_uuid: "62e127c1a09e4b0c8f757cff1578528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e577d9c828224c7eb6b7c9686c61967e"
I20260812 06:51:13.985942 19187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cbebe4141940a8ac0fa155584e0107" candidate_uuid: "62e127c1a09e4b0c8f757cff1578528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60b24344a0645d4a4a07961ca15533b"
I20260812 06:51:13.986006 19417 raft_consensus.cc:3037] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:13.986133 19187 raft_consensus.cc:3037] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:13.988339 19417 raft_consensus.cc:2454] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e127c1a09e4b0c8f757cff1578528c in term 1.
I20260812 06:51:13.988372 19187 raft_consensus.cc:2454] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e127c1a09e4b0c8f757cff1578528c in term 1.
I20260812 06:51:13.988682 18844 leader_election.cc:302] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e127c1a09e4b0c8f757cff1578528c, e577d9c828224c7eb6b7c9686c61967e; no voters:
I20260812 06:51:13.988945 19543 raft_consensus.cc:2781] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:13.989324 19543 raft_consensus.cc:686] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [term 1 LEADER]: Becoming Leader. State: Replica: 62e127c1a09e4b0c8f757cff1578528c, State: Running, Role: LEADER
I20260812 06:51:13.989770 19543 consensus_queue.cc:237] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } }
I20260812 06:51:13.993774 18703 catalog_manager.cc:5495] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c reported cstate change: term changed from 0 to 1, leader changed from <none> to 62e127c1a09e4b0c8f757cff1578528c (127.17.69.65). New cstate: current_term: 1 leader_uuid: "62e127c1a09e4b0c8f757cff1578528c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e577d9c828224c7eb6b7c9686c61967e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e60b24344a0645d4a4a07961ca15533b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 33801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e127c1a09e4b0c8f757cff1578528c" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36611 } health_report { overall_health: HEALTHY } } }
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:14.025122 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.012223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.17.69.65:36611)
0812 06:51:14.012443 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.012451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.012823 (+ 372us) server_negotiation.cc:407] Connection header received
0812 06:51:14.012940 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.012944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.012999 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.013075 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.014486 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.015182 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.016321 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.016683 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.016768 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.020347 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.020359 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.020373 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.020403 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.022950 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.023539 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.023545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.023548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.023691 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.023948 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.023951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.023953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.024274 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.024484 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.024664 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.024825 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:14.073827 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.059426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.17.69.65:36611)
0812 06:51:14.059634 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.059642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.059951 (+ 309us) server_negotiation.cc:407] Connection header received
0812 06:51:14.060135 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.060140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.060203 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.060286 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.061648 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.062449 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.063705 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.064122 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.064233 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.067848 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.067854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.067858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.067900 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.071012 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.071612 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.071618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.071636 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.071812 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.072437 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.072443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.072447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.072731 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.073019 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.073276 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.073495 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 06:51:14.076138 19507 tablet.cc:2369] T 49cbebe4141940a8ac0fa155584e0107 P e577d9c828224c7eb6b7c9686c61967e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:51:14.089424 19274 tablet.cc:2369] T 49cbebe4141940a8ac0fa155584e0107 P e60b24344a0645d4a4a07961ca15533b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:14.118916 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.105826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.17.69.65:36611)
0812 06:51:14.105993 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.105999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.106327 (+ 328us) server_negotiation.cc:407] Connection header received
0812 06:51:14.106516 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.106520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.106596 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.106680 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.107980 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.108838 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.110078 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.110525 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.110643 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.114191 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.114196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.114199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.114231 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.116494 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.117080 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.117085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.117087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.117228 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.117564 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.117568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.117570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.117824 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.118064 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.118375 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.118580 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 06:51:14.121172 18885 tablet_service.cc:2908] Rejecting scan request for tablet 49cbebe4141940a8ac0fa155584e0107: Uninitialized: clean time has not yet been initialized
W20260812 06:51:14.123540 19037 tablet.cc:2369] T 49cbebe4141940a8ac0fa155584e0107 P 62e127c1a09e4b0c8f757cff1578528c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:14.125056 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18811
I20260812 06:51:14.136334 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.125751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.17.69.65:36611)
0812 06:51:14.125911 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.125917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.126193 (+ 276us) server_negotiation.cc:407] Connection header received
0812 06:51:14.126313 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.126317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.126408 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.126483 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.127493 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.128312 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.129392 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.129771 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.129853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.131729 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.131734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.131737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.131771 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.134091 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.134766 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.134770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.134773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.134900 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.135205 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.135209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.135211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.135462 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.135650 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.135841 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.136016 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:51:14.136919 19027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:14.226466 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18811
I20260812 06:51:14.250063 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19049
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:14.263917 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.250830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56162 (local address 127.17.69.66:33801)
0812 06:51:14.251011 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.251017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.251036 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:51:14.251218 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.251224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.251298 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.251395 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.252586 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.253353 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.254746 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.255127 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.255220 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.258944 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.258958 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.258972 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.259005 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.261561 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.262193 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.262200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.262202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.262363 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.262647 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.262650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.262652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.263028 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.263284 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.263453 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.263607 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:51:14.264503 19261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:14.349179 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19049
I20260812 06:51:14.371786 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19280
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[18628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:14.386577 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:14.372535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.17.69.67:40875)
0812 06:51:14.372707 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:14.372711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:14.372731 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:51:14.372855 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:14.372859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:14.372933 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:14.373015 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:14.374185 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.374971 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.376302 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:14.376823 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:14.376934 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:14.381104 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:14.381119 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:14.381135 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:14.381173 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:14.383856 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.384501 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.384509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.384511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.384633 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:14.384996 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:14.385000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:14.385002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:14.385351 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:14.385624 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:14.385943 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:51:14.386147 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:51:14.387203 19493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:14.472661 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19280
I20260812 06:51:14.494534 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18648
I20260812 06:51:14.495630 18754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:14.602234 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18648
2026-08-12T06:51:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4093 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[19574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[19574](info): set up 2 sockets
Aug 12 06:51:14 dist-test-slave-6zbq krb5kdc[19574](info): commencing operation
krb5kdc: starting...
W20260812 06:51:15.823760 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:15 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:16.002487 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34121
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:34121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:16.312170 19603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:16.312441 19603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:16.312556 19603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:16.319577 19603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:16.319665 19603 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:51:16.319736 19603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:16.319818 19603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:16.319885 19603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:16.319969 19603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:16.329411 19603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:34121
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:16.330610 19603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:16.331920 19603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:16.342828 19616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:16.342777 19624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:16.342801 19620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:16.343641 19603 server_base.cc:1034] running on GCE node
I20260812 06:51:16.344354 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:16.345633 19603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:16.346830 19603 hybrid_clock.cc:648] HybridClock initialized: now 1786517476346766 us; error 41 us; skew 500 ppm
I20260812 06:51:16.347103 19603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:16 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517476, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:16.349959 19603 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:16.351276 19603 webserver.cc:460] Webserver started at http://127.17.69.126:46585/ using document root <none> and password file <none>
I20260812 06:51:16.351898 19603 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:16.351996 19603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:16.352298 19603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:16.355098 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "451af43be29543ecbdf5fb8b783970d9"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "d18ccf7d5bc1740a9699e0744460900f"
server_key_iv: "5a8da66ff4e9eaddcbda58b56dbaa5e4"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.355810 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "451af43be29543ecbdf5fb8b783970d9"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "d18ccf7d5bc1740a9699e0744460900f"
server_key_iv: "5a8da66ff4e9eaddcbda58b56dbaa5e4"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.360512 19603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 06:51:16.363891 19636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:16.365126 19603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:51:16.365298 19603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "451af43be29543ecbdf5fb8b783970d9"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "d18ccf7d5bc1740a9699e0744460900f"
server_key_iv: "5a8da66ff4e9eaddcbda58b56dbaa5e4"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.365494 19603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:16.375335 19603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:16.378793 19603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:16.379011 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:16.399344 19603 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:34121
I20260812 06:51:16.399364 19756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:34121 every 8 connection(s)
I20260812 06:51:16.400699 19603 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:16.404265 19759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:16.406992 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19603
I20260812 06:51:16.407224 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:16.408032 17685 external_mini_cluster.cc:1428] Setting key fba6e55771eb5e20bcb3ca5e6e4aba25
I20260812 06:51:16.415800 19759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Bootstrap starting.
I20260812 06:51:16.418619 19759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:16.419538 19759 log.cc:824] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:51:16 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:16.422636 19759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: No bootstrap required, opened a new log
I20260812 06:51:16.429502 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:16.410166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.17.69.126:34121)
0812 06:51:16.410913 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:16.410929 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:16.410969 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:51:16.412614 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:16.412634 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:16.413015 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:16.413370 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:16.414771 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.416068 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.417244 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.417889 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.418046 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:16.422526 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:16.422534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:16.422547 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:16.422580 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:16.425516 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.426316 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.426323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.426328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.426583 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.426978 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.426981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.426983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.427517 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:16.427751 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:16.428174 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 06:51:16.428507 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":332,"threads_started":1}
I20260812 06:51:16.430744 19759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } }
I20260812 06:51:16.431069 19759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:16.431173 19759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451af43be29543ecbdf5fb8b783970d9, State: Initialized, Role: FOLLOWER
I20260812 06:51:16.431695 19759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } }
I20260812 06:51:16.431918 19759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:16.432060 19759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:16.432262 19759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:16.434209 19759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } }
I20260812 06:51:16.434687 19759 leader_election.cc:302] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 451af43be29543ecbdf5fb8b783970d9; no voters:
I20260812 06:51:16.435318 19759 leader_election.cc:288] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:16.435420 19770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:16.436053 19770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [term 1 LEADER]: Becoming Leader. State: Replica: 451af43be29543ecbdf5fb8b783970d9, State: Running, Role: LEADER
I20260812 06:51:16.436575 19770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } }
I20260812 06:51:16.437108 19759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:16.438948 19772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "451af43be29543ecbdf5fb8b783970d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } } }
I20260812 06:51:16.439261 19772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:16.439731 19775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 451af43be29543ecbdf5fb8b783970d9. Latest consensus state: current_term: 1 leader_uuid: "451af43be29543ecbdf5fb8b783970d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451af43be29543ecbdf5fb8b783970d9" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34121 } } }
I20260812 06:51:16.440068 19781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:16.440074 19775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:16.445535 19781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:16.453074 19781 catalog_manager.cc:1347] Generated new cluster ID: a81526554c754851b5d4641867a6c962
I20260812 06:51:16.453197 19781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:16.465116 19781 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:16.466845 19781 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:16.476111 19781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Generated new TSK 0
I20260812 06:51:16.476981 19781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:16.531360 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34121
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:16.846855 19808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:16.847129 19808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:16.847265 19808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:16.854231 19808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:16.854354 19808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:16.854473 19808 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:16.854560 19808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:16.864114 19808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:16.865271 19808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:16.866561 19808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:16.878953 19813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:16.878976 19821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:16.879112 19818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:16.880113 19808 server_base.cc:1034] running on GCE node
I20260812 06:51:16.880656 19808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:16.881510 19808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:16.882710 19808 hybrid_clock.cc:648] HybridClock initialized: now 1786517476882664 us; error 79 us; skew 500 ppm
I20260812 06:51:16.882973 19808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:16 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517476, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:16.885922 19808 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:16.887290 19808 webserver.cc:460] Webserver started at http://127.17.69.65:38465/ using document root <none> and password file <none>
I20260812 06:51:16.887916 19808 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:16.888015 19808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:16.888312 19808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:16.891043 19808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "b1e4002303364188ab1eba00ee335317"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "15b2383ba451a4cdc071582f6bf3df21"
server_key_iv: "9bfe21254e44fa01daae4c8d4dd1b1cf"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.891800 19808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "b1e4002303364188ab1eba00ee335317"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "15b2383ba451a4cdc071582f6bf3df21"
server_key_iv: "9bfe21254e44fa01daae4c8d4dd1b1cf"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.896492 19808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:16.899631 19835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:16.900799 19808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:16.900935 19808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "b1e4002303364188ab1eba00ee335317"
format_stamp: "Formatted at 2026-08-12 06:51:16 on dist-test-slave-6zbq"
server_key: "15b2383ba451a4cdc071582f6bf3df21"
server_key_iv: "9bfe21254e44fa01daae4c8d4dd1b1cf"
server_key_version: "encryptionkey@0"
I20260812 06:51:16.901132 19808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:16.923420 19808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:16.927235 19808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:16.927467 19808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:16.929982 19808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:16.930063 19808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:16.930202 19808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:16.930272 19808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:16 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517476, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:16.953583 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:16.930831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:45533 (local address 127.17.69.126:34121)
0812 06:51:16.931327 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:16.931334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:16.932464 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 06:51:16.935230 (+ 2766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:16.935236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:16.935339 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:16.935477 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:16.937880 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.939131 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.941844 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.942434 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.942574 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:16.946963 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:16.946971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:16.946977 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:16.947024 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:16.950214 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.951084 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.951091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.951095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.951249 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.951775 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.951781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.951784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.952164 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:16.952333 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:16.952987 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 06:51:16.953228 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260812 06:51:16.954761 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:16.931474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.65:45533)
0812 06:51:16.932285 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:16.932336 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:16.934900 (+ 2564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:16.935749 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:16.935760 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:16.936565 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:16.937628 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:16.937657 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.939355 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.939360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:16.941673 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:16.941692 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.942689 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.942702 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:16.942919 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:16.943571 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:16.943605 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:16.946751 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:16.950379 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:16.950388 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:16.950403 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:16.950954 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:16.951375 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:16.951379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:16.951383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:16.951648 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:16.952308 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:16.952316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:16.952641 (+ 325us) client_negotiation.cc:769] Sending connection context
0812 06:51:16.952859 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 06:51:16.953314 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":316,"threads_started":1}
I20260812 06:51:16.973305 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:16.960387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:50791 (local address 127.17.69.126:34121)
0812 06:51:16.960527 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:16.960531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:16.961175 (+ 644us) server_negotiation.cc:407] Connection header received
0812 06:51:16.961324 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:16.961330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:16.961396 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:16.961472 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:16.962732 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.963654 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.964919 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.965359 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.965466 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:16.967412 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:16.967418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:16.967423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:16.967458 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:16.970434 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.971246 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.971251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.971255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.971413 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:16.971886 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:16.971891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:16.971894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:16.972237 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:16.972373 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:16.972780 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 06:51:16.972983 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 06:51:16.973474 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:16.960482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.65:50791)
0812 06:51:16.961051 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:16.961096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:16.961220 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:16.961511 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:16.961516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:16.962108 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:16.962592 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:16.962606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.963809 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.963814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:16.964788 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:16.964800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:16.965541 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:16.965548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:16.965701 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:16.966211 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:16.966227 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:16.967185 (+ 958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:16.970584 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:16.970588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:16.970592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:16.971120 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:16.971566 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:16.971571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:16.971574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:16.971760 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:16.972357 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:16.972362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:16.972546 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 06:51:16.972663 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:51:16.972977 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":313,"threads_started":1}
I20260812 06:51:16.985503 19808 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:43609
I20260812 06:51:16.985526 20053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:43609 every 8 connection(s)
I20260812 06:51:16.986596 19808 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:16.988349 20054 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34121
I20260812 06:51:16.988585 20054 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:16.988646 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19808
I20260812 06:51:16.988865 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:16.989301 20054 heartbeater.cc:511] Master 127.17.69.126:34121 requested a full tablet report, sending...
I20260812 06:51:16.989696 17685 external_mini_cluster.cc:1428] Setting key 3f9812118e7b8ee7ea5b720541d9f50b
I20260812 06:51:16.991321 19663 ts_manager.cc:194] Registered new tserver with Master: b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609)
I20260812 06:51:16.992727 19663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:45533
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:17.029639 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34121
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:17.334586 20060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:17.334838 20060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:17.334972 20060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:17.341794 20060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:17.341884 20060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:17.342013 20060 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:17.342115 20060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:17.351629 20060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:17.352777 20060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:17.354009 20060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:17.365980 20074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:17.366062 20071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:17.366031 20076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:17.366973 20060 server_base.cc:1034] running on GCE node
I20260812 06:51:17.367523 20060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:17.368366 20060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:17.369588 20060 hybrid_clock.cc:648] HybridClock initialized: now 1786517477369516 us; error 41 us; skew 500 ppm
I20260812 06:51:17.369866 20060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:17.376778 20060 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:17.378054 20060 webserver.cc:460] Webserver started at http://127.17.69.66:38633/ using document root <none> and password file <none>
I20260812 06:51:17.378722 20060 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:17.378808 20060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:17.379074 20060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:17.381789 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "a3823bd04326435d8a3f0a463d0d2385"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "c7c13cee51f7417628e137451ca4c3c0"
server_key_iv: "907ee97ae50520cf6f85f49230ac1fac"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.382735 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "a3823bd04326435d8a3f0a463d0d2385"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "c7c13cee51f7417628e137451ca4c3c0"
server_key_iv: "907ee97ae50520cf6f85f49230ac1fac"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.387526 20060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:17.390689 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:17.391902 20060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:51:17.392050 20060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "a3823bd04326435d8a3f0a463d0d2385"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "c7c13cee51f7417628e137451ca4c3c0"
server_key_iv: "907ee97ae50520cf6f85f49230ac1fac"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.392239 20060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:17.403949 20060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:17.407843 20060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:17.408072 20060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:17.410650 20060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:17.410750 20060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:17.410889 20060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:17.410964 20060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:17.434722 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.411844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.66:53837)
0812 06:51:17.412708 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:17.412753 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:17.414730 (+ 1977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:17.415490 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:17.415498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:17.416266 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:17.417169 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.417199 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.418654 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.418659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.421155 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.421175 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.422104 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.422112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.422438 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.423266 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:17.423300 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:17.426617 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:17.430181 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.430193 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.430208 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.430712 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.431191 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.431199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.431203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.431417 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.432081 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:17.432088 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:17.432436 (+ 348us) client_negotiation.cc:769] Sending connection context
0812 06:51:17.432764 (+ 328us) client_negotiation.cc:240] Negotiation successful
0812 06:51:17.433254 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":365,"threads_started":1}
I20260812 06:51:17.435607 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.411239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:53837 (local address 127.17.69.126:34121)
0812 06:51:17.411428 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:17.411435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:17.412871 (+ 1436us) server_negotiation.cc:407] Connection header received
0812 06:51:17.415033 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:17.415040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:17.415132 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:17.415245 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:17.417431 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.418475 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.421319 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.421834 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.421965 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.426867 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:17.426874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:17.426878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:17.426917 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:17.430037 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.430844 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.430851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.430854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.431048 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.431544 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.431549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.431553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.431826 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:17.431992 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:17.435231 (+ 3239us) server_negotiation.cc:299] Negotiation successful
0812 06:51:17.435394 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:51:17.454991 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.440347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:53383 (local address 127.17.69.126:34121)
0812 06:51:17.440518 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:17.440523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:17.441207 (+ 684us) server_negotiation.cc:407] Connection header received
0812 06:51:17.441384 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:17.441390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:17.441464 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:17.441564 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:17.442732 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.443721 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.445692 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.446233 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.446387 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.448645 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:17.448652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:17.448657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:17.448694 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:17.452166 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.452878 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.452884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.452887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.453039 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.453432 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.453437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.453440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.453868 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:17.454007 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:17.454438 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 06:51:17.454668 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:51:17.455102 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.440407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.66:53383)
0812 06:51:17.441092 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:17.441113 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:17.441262 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:17.441587 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:17.441592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:17.442153 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:17.442599 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.442614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.443870 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.443878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.445527 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.445545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.446588 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.446598 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.446796 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.447453 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:17.447470 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:17.448433 (+ 963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:17.452322 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.452330 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.452333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.452759 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.453170 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.453173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.453176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.453305 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.453992 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:17.453997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:17.454183 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 06:51:17.454306 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 06:51:17.454659 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":339,"threads_started":1}
I20260812 06:51:17.456053 20060 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:44169
I20260812 06:51:17.456063 20295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:44169 every 8 connection(s)
I20260812 06:51:17.457549 20060 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:17.459697 20296 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34121
I20260812 06:51:17.459895 20296 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:17.460445 20296 heartbeater.cc:511] Master 127.17.69.126:34121 requested a full tablet report, sending...
I20260812 06:51:17.461633 19663 ts_manager.cc:194] Registered new tserver with Master: a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:17.462443 19663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:53837
I20260812 06:51:17.463913 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20060
I20260812 06:51:17.464125 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:17.464676 17685 external_mini_cluster.cc:1428] Setting key edeb16c47bdd6b5c02cb1d6f368ee9ea
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:17.505790 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34121
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:17.814039 20307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:17.814291 20307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:17.814424 20307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:17.821241 20307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:17.821329 20307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:17.821415 20307 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:17.821491 20307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:17.830633 20307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:17.831712 20307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:17.832944 20307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:17.844825 20320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:17.844858 20325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:17.844763 20322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:17.845700 20307 server_base.cc:1034] running on GCE node
I20260812 06:51:17.846191 20307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:17.847007 20307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:17.848212 20307 hybrid_clock.cc:648] HybridClock initialized: now 1786517477848145 us; error 37 us; skew 500 ppm
I20260812 06:51:17.848464 20307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:17.851305 20307 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:17.852591 20307 webserver.cc:460] Webserver started at http://127.17.69.67:39701/ using document root <none> and password file <none>
I20260812 06:51:17.853190 20307 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:17.853296 20307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:17.853579 20307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:17.856218 20307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "3d68f6f863e647349aa6c201da4c5d5b"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "7f8fc321220e316fea52fba0ce54936a"
server_key_iv: "dfcb0838ee9e4184c704307bb211ede4"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.856950 20307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "3d68f6f863e647349aa6c201da4c5d5b"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "7f8fc321220e316fea52fba0ce54936a"
server_key_iv: "dfcb0838ee9e4184c704307bb211ede4"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.861529 20307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 06:51:17.864578 20344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:17.865708 20307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:17.865851 20307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "3d68f6f863e647349aa6c201da4c5d5b"
format_stamp: "Formatted at 2026-08-12 06:51:17 on dist-test-slave-6zbq"
server_key: "7f8fc321220e316fea52fba0ce54936a"
server_key_iv: "dfcb0838ee9e4184c704307bb211ede4"
server_key_version: "encryptionkey@0"
I20260812 06:51:17.866031 20307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:17.889906 20307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:17.893270 20307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:17.893500 20307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:17.896153 20307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:17.896297 20307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 06:51:17.896476 20307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:17.896549 20307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:17.917421 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.896426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:41615 (local address 127.17.69.126:34121)
0812 06:51:17.896625 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:17.896631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:17.897953 (+ 1322us) server_negotiation.cc:407] Connection header received
0812 06:51:17.899842 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:17.899848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:17.899927 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:17.900012 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:17.901872 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.903038 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.905414 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.905837 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.905948 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.910611 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:17.910617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:17.910622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:17.910665 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:17.913510 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.914557 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.914563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.914567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.914744 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.915342 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.915347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.915350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.915702 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:17.915871 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:17.916827 (+ 956us) server_negotiation.cc:299] Negotiation successful
0812 06:51:17.917062 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:51:17.918795 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.896909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.67:41615)
0812 06:51:17.897781 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:17.897829 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:17.899577 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:17.900230 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:17.900237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:17.900886 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:17.901655 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.901693 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.903226 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.903232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.905274 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.905287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.906046 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.906054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.906383 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.907037 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:17.907070 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:17.910375 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:17.913652 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.913661 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.913679 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.914428 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.914868 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.914877 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.914880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.915188 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.915862 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:17.915871 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:17.916352 (+ 481us) client_negotiation.cc:769] Sending connection context
0812 06:51:17.916671 (+ 319us) client_negotiation.cc:240] Negotiation successful
0812 06:51:17.917192 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":387,"threads_started":1}
I20260812 06:51:17.934523 20307 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:42359
I20260812 06:51:17.934549 20553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:42359 every 8 connection(s)
I20260812 06:51:17.935685 20307 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:17.938449 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.924998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:52451 (local address 127.17.69.126:34121)
0812 06:51:17.925150 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:17.925155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:17.925990 (+ 835us) server_negotiation.cc:407] Connection header received
0812 06:51:17.926111 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:17.926117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:17.926194 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:17.926307 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:17.927326 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.928366 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.930567 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.931102 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.931212 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.932790 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:17.932797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:17.932801 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:17.932839 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:17.935575 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.936327 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.936332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.936336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.936486 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.936926 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.936932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.936936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.937276 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:17.937416 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:17.937871 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 06:51:17.938077 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:51:17.937668 20555 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34121
I20260812 06:51:17.938498 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.925143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.17.69.67:52451)
0812 06:51:17.925892 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:17.925909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:17.926012 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:17.926322 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:17.926325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:17.926768 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:17.927174 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.927187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.928535 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.928544 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.930414 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:17.930434 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.931288 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.931294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:17.931434 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.931971 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:17.931986 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:17.932607 (+ 621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:17.935696 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.935702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.935705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.936189 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.936603 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:17.936608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:17.936611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:17.936784 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:17.937380 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:17.937385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:17.937590 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 06:51:17.937771 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 06:51:17.938072 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":449,"threads_started":1}
I20260812 06:51:17.938905 20555 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:17.939567 20555 heartbeater.cc:511] Master 127.17.69.126:34121 requested a full tablet report, sending...
I20260812 06:51:17.941020 19663 ts_manager.cc:194] Registered new tserver with Master: 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67:42359)
I20260812 06:51:17.942045 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20307
I20260812 06:51:17.942073 19663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:41615
I20260812 06:51:17.942265 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:17.942694 17685 external_mini_cluster.cc:1428] Setting key 55a5e90b08241b45c078d18ae47eb940
I20260812 06:51:17.945416 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:51:17 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:17.992236 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:17.979478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.17.69.126:34121)
0812 06:51:17.979690 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:17.979697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:17.979975 (+ 278us) server_negotiation.cc:407] Connection header received
0812 06:51:17.980112 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:17.980116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:17.980191 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:17.980281 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:17.981516 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.982211 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.983415 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:17.983836 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:17.983926 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:17.987409 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:17.987414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:17.987417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:17.987449 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:17.989840 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.990435 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.990440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.990442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.990564 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:17.990920 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:17.990923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:17.990925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:17.991194 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:17.991423 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:17.991715 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 06:51:17.991928 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:51:17.995537 20054 heartbeater.cc:503] Master 127.17.69.126:34121 was elected leader, sending a full tablet report...
I20260812 06:51:17.996042 19663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:18.002938 19663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:18.033516 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.015032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.17.69.65:43609)
0812 06:51:18.025598 (+ 10566us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.025606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.026524 (+ 918us) server_negotiation.cc:407] Connection header received
0812 06:51:18.026823 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.026830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.027030 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.027197 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.027926 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.029034 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.031626 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.032497 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.032617 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.032822 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.033069 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.033235 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10402,"thread_start_us":10272,"threads_started":1}
I20260812 06:51:18.033622 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.014275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:43609 (local address 127.0.0.1:39912)
0812 06:51:18.026406 (+ 12131us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.026424 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.026636 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.027449 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.027453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.027475 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.027773 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.027780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.029230 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.029236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.031494 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.031504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.032691 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.032698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.032872 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.032910 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.033020 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.033203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11997,"mutex_wait_us":52,"thread_start_us":9891,"threads_started":1}
I20260812 06:51:18.034022 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.013882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:42359 (local address 127.0.0.1:37082)
0812 06:51:18.024589 (+ 10707us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.024651 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.024824 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.027005 (+ 2181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.027009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.027033 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.027461 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.027472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.029106 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.029111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.031189 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.031205 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.033191 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.033197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.033386 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.033418 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.033505 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.033641 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10422,"mutex_wait_us":169,"thread_start_us":11096,"threads_started":1}
I20260812 06:51:18.034808 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.014025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.17.69.67:42359)
0812 06:51:18.026126 (+ 12101us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.026134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.026169 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:51:18.026265 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.026272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.026507 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.026709 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.027664 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.028975 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.031360 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.032687 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.032825 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.033051 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.034314 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.034491 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11944,"thread_start_us":11786,"threads_started":1}
I20260812 06:51:18.034914 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.013945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:44169 (local address 127.0.0.1:43386)
0812 06:51:18.025093 (+ 11148us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.025110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.025221 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.026504 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.026511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.026542 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.026947 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.026962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.028565 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.028571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.031158 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.031178 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.034157 (+ 2979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.034164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.034374 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.034409 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.034480 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.034592 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11028,"mutex_wait_us":135,"thread_start_us":12197,"threads_started":1}
I20260812 06:51:18.034988 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.014674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.17.69.66:44169)
0812 06:51:18.025655 (+ 10981us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.025664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.025695 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:51:18.025810 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.025819 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.026063 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.026246 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.027164 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.028427 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.031328 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.033698 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.033829 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.034011 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.034556 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.034716 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10838,"thread_start_us":10774,"threads_started":1}
I20260812 06:51:18.037169 19931 tablet_service.cc:1467] Processing CreateTablet for tablet 729c191696b044ec91f7a8ccfaa68aff (DEFAULT_TABLE table=test-table [id=f3f50e5369624ed5a12f8eddca43fb02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.038419 19931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729c191696b044ec91f7a8ccfaa68aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.038157 20437 tablet_service.cc:1467] Processing CreateTablet for tablet 729c191696b044ec91f7a8ccfaa68aff (DEFAULT_TABLE table=test-table [id=f3f50e5369624ed5a12f8eddca43fb02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.038156 20178 tablet_service.cc:1467] Processing CreateTablet for tablet 729c191696b044ec91f7a8ccfaa68aff (DEFAULT_TABLE table=test-table [id=f3f50e5369624ed5a12f8eddca43fb02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.039603 20437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729c191696b044ec91f7a8ccfaa68aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.039800 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729c191696b044ec91f7a8ccfaa68aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.049324 20587 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Bootstrap starting.
I20260812 06:51:18.052834 20587 tablet_bootstrap.cc:654] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.053431 20589 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Bootstrap starting.
I20260812 06:51:18.054175 20587 log.cc:824] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:18.054301 20591 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Bootstrap starting.
I20260812 06:51:18.056967 20589 tablet_bootstrap.cc:654] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.057126 20591 tablet_bootstrap.cc:654] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.058267 20591 log.cc:824] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:18.058817 20589 log.cc:824] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:18.060855 20591 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: No bootstrap required, opened a new log
I20260812 06:51:18.061093 20591 ts_tablet_manager.cc:1397] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:51:18.061200 20587 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: No bootstrap required, opened a new log
I20260812 06:51:18.061455 20587 ts_tablet_manager.cc:1397] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260812 06:51:18.061522 20589 tablet_bootstrap.cc:492] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: No bootstrap required, opened a new log
I20260812 06:51:18.061780 20589 ts_tablet_manager.cc:1397] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 06:51:18.067910 20591 raft_consensus.cc:348] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.068295 20591 raft_consensus.cc:374] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.068382 20591 raft_consensus.cc:729] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d68f6f863e647349aa6c201da4c5d5b, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.068945 20591 consensus_queue.cc:260] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.070076 20555 heartbeater.cc:503] Master 127.17.69.126:34121 was elected leader, sending a full tablet report...
I20260812 06:51:18.071677 20591 ts_tablet_manager.cc:1428] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260812 06:51:18.071628 20589 raft_consensus.cc:348] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.072063 20589 raft_consensus.cc:374] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.072158 20589 raft_consensus.cc:729] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3823bd04326435d8a3f0a463d0d2385, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.071993 20587 raft_consensus.cc:348] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.072391 20587 raft_consensus.cc:374] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.072516 20587 raft_consensus.cc:729] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e4002303364188ab1eba00ee335317, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.072736 20589 consensus_queue.cc:260] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.073079 20587 consensus_queue.cc:260] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.073734 20296 heartbeater.cc:503] Master 127.17.69.126:34121 was elected leader, sending a full tablet report...
I20260812 06:51:18.074419 20589 ts_tablet_manager.cc:1428] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 06:51:18.075819 20587 ts_tablet_manager.cc:1428] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260812 06:51:18.126632 20600 raft_consensus.cc:482] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:18.126894 20600 raft_consensus.cc:504] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.128450 20600 leader_election.cc:288] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67:42359), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.134917 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.129041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:55409 (local address 127.17.69.66:44169)
0812 06:51:18.129244 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.129250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.129983 (+ 733us) server_negotiation.cc:407] Connection header received
0812 06:51:18.130144 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.130148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.130200 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.130272 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.130901 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.131593 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.133280 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.134004 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.134157 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.134265 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.134565 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.134685 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:51:18.134970 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.128864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:35549 (local address 127.17.69.67:42359)
0812 06:51:18.129058 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.129064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.129815 (+ 751us) server_negotiation.cc:407] Connection header received
0812 06:51:18.129994 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.129998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.130055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.130126 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.130720 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.131414 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.133183 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.134018 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.134185 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.134283 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.134616 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.134730 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:51:18.134991 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.128889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:44169 (local address 127.17.69.65:55409)
0812 06:51:18.129885 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.129899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.130022 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.130322 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.130327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.130378 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.130756 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.130765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.131732 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.131742 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.133123 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.133134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.134266 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.134273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.134418 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.134442 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.134522 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.134640 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":658,"threads_started":1}
I20260812 06:51:18.135087 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.128825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:42359 (local address 127.17.69.65:35549)
0812 06:51:18.129704 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.129722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.129856 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.130153 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.130157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.130181 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.130572 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.130582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.131538 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.131543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.133017 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.133030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.134318 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.134325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.134471 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.134494 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.134575 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.134701 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":1,"thread_start_us":465,"threads_started":1}
I20260812 06:51:18.136088 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729c191696b044ec91f7a8ccfaa68aff" candidate_uuid: "b1e4002303364188ab1eba00ee335317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385" is_pre_election: true
I20260812 06:51:18.136235 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729c191696b044ec91f7a8ccfaa68aff" candidate_uuid: "b1e4002303364188ab1eba00ee335317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d68f6f863e647349aa6c201da4c5d5b" is_pre_election: true
I20260812 06:51:18.136687 20213 raft_consensus.cc:2454] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1e4002303364188ab1eba00ee335317 in term 0.
I20260812 06:51:18.136695 20470 raft_consensus.cc:2454] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1e4002303364188ab1eba00ee335317 in term 0.
I20260812 06:51:18.137389 19837 leader_election.cc:302] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, b1e4002303364188ab1eba00ee335317; no voters:
I20260812 06:51:18.137759 20600 raft_consensus.cc:2781] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:18.137884 20600 raft_consensus.cc:482] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:18.137981 20600 raft_consensus.cc:3037] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.140092 20600 raft_consensus.cc:504] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.140697 20600 leader_election.cc:288] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [CANDIDATE]: Term 1 election: Requested vote from peers 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67:42359), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.141043 20470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729c191696b044ec91f7a8ccfaa68aff" candidate_uuid: "b1e4002303364188ab1eba00ee335317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d68f6f863e647349aa6c201da4c5d5b"
I20260812 06:51:18.141228 20470 raft_consensus.cc:3037] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.141191 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729c191696b044ec91f7a8ccfaa68aff" candidate_uuid: "b1e4002303364188ab1eba00ee335317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385"
I20260812 06:51:18.141383 20213 raft_consensus.cc:3037] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.143625 20470 raft_consensus.cc:2454] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1e4002303364188ab1eba00ee335317 in term 1.
I20260812 06:51:18.143837 20213 raft_consensus.cc:2454] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1e4002303364188ab1eba00ee335317 in term 1.
I20260812 06:51:18.143956 19837 leader_election.cc:302] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, b1e4002303364188ab1eba00ee335317; no voters:
I20260812 06:51:18.144218 20600 raft_consensus.cc:2781] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:18.144619 20600 raft_consensus.cc:686] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 1 LEADER]: Becoming Leader. State: Replica: b1e4002303364188ab1eba00ee335317, State: Running, Role: LEADER
I20260812 06:51:18.145083 20600 consensus_queue.cc:237] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } }
I20260812 06:51:18.149187 19663 catalog_manager.cc:5495] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1e4002303364188ab1eba00ee335317 (127.17.69.65). New cstate: current_term: 1 leader_uuid: "b1e4002303364188ab1eba00ee335317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } health_report { overall_health: HEALTHY } } }
W20260812 06:51:18.188066 20556 tablet.cc:2369] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:18.199461 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.194254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.17.69.65:43609)
0812 06:51:18.194491 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.194497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.194515 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:51:18.194831 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.194835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.194910 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.195015 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:18.195444 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.196176 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.197503 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.197887 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.197973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.198423 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.198932 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.199036 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.199150 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.199213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:51:18.208046 20213 raft_consensus.cc:1261] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Refusing update from remote peer b1e4002303364188ab1eba00ee335317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:18.209134 20605 consensus_queue.cc:1035] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.209517 20470 raft_consensus.cc:1261] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Refusing update from remote peer b1e4002303364188ab1eba00ee335317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 06:51:18.209738 20297 tablet.cc:2369] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:18.210318 20605 consensus_queue.cc:1035] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.219794 20614 mvcc.cc:205] Tried to move back new op lower bound from 7317575590728024064 to 7317575590485291008. Current Snapshot: MvccSnapshot[applied={T|T < 7317575590728024064}]
I20260812 06:51:18.227753 20616 mvcc.cc:205] Tried to move back new op lower bound from 7317575590728024064 to 7317575590485291008. Current Snapshot: MvccSnapshot[applied={T|T < 7317575590728024064}]
I20260812 06:51:18.241014 19663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:18.241322 19663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:18.244915 19663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Sending DeleteTablet for 3 replicas of tablet 729c191696b044ec91f7a8ccfaa68aff
I20260812 06:51:18.246182 19931 tablet_service.cc:1514] Processing DeleteTablet for tablet 729c191696b044ec91f7a8ccfaa68aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39912
I20260812 06:51:18.246176 20178 tablet_service.cc:1514] Processing DeleteTablet for tablet 729c191696b044ec91f7a8ccfaa68aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:43386
I20260812 06:51:18.246168 20437 tablet_service.cc:1514] Processing DeleteTablet for tablet 729c191696b044ec91f7a8ccfaa68aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:37082
I20260812 06:51:18.250514 19663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:18.251196 19663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:18.259042 20435 tablet_service.cc:1467] Processing CreateTablet for tablet 81ead805e8fb4b938910457625e4480d (DEFAULT_TABLE table=test-table [id=344e11da7a5f4247b292c02043924b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.259269 19922 tablet_service.cc:1467] Processing CreateTablet for tablet 81ead805e8fb4b938910457625e4480d (DEFAULT_TABLE table=test-table [id=344e11da7a5f4247b292c02043924b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.259665 20435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ead805e8fb4b938910457625e4480d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.259809 19922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ead805e8fb4b938910457625e4480d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.261654 20634 tablet_replica.cc:330] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: stopping tablet replica
I20260812 06:51:18.262307 20634 raft_consensus.cc:2229] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:18.262939 20634 raft_consensus.cc:2258] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.263311 20177 tablet_service.cc:1467] Processing CreateTablet for tablet 81ead805e8fb4b938910457625e4480d (DEFAULT_TABLE table=test-table [id=344e11da7a5f4247b292c02043924b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:18.263862 20177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ead805e8fb4b938910457625e4480d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.265662 20634 ts_tablet_manager.cc:1905] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.266264 20591 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Bootstrap starting.
I20260812 06:51:18.267976 20591 tablet_bootstrap.cc:654] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.272791 20589 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Bootstrap starting.
I20260812 06:51:18.272910 20634 ts_tablet_manager.cc:1918] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:18.273102 20634 log.cc:1196] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/729c191696b044ec91f7a8ccfaa68aff
I20260812 06:51:18.273710 20634 ts_tablet_manager.cc:1939] T 729c191696b044ec91f7a8ccfaa68aff P 3d68f6f863e647349aa6c201da4c5d5b: Deleting consensus metadata
I20260812 06:51:18.274459 20589 tablet_bootstrap.cc:654] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.275571 19638 catalog_manager.cc:4841] TS 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67:42359): tablet 729c191696b044ec91f7a8ccfaa68aff (table test-table [id=f3f50e5369624ed5a12f8eddca43fb02]) successfully deleted
I20260812 06:51:18.276105 20587 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Bootstrap starting.
I20260812 06:51:18.277809 20587 tablet_bootstrap.cc:654] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.280747 20632 tablet_replica.cc:330] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: stopping tablet replica
I20260812 06:51:18.281262 20632 raft_consensus.cc:2229] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:18.281356 20591 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: No bootstrap required, opened a new log
I20260812 06:51:18.281535 20591 ts_tablet_manager.cc:1397] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260812 06:51:18.282404 20591 raft_consensus.cc:348] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.282719 20591 raft_consensus.cc:374] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.282883 20591 raft_consensus.cc:729] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d68f6f863e647349aa6c201da4c5d5b, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.281786 20632 raft_consensus.cc:2258] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.283217 20591 consensus_queue.cc:260] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.284890 20591 ts_tablet_manager.cc:1428] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:18.286736 20631 tablet_replica.cc:330] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: stopping tablet replica
I20260812 06:51:18.287446 20631 raft_consensus.cc:2229] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:18.288355 20631 raft_consensus.cc:2258] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.288468 20632 ts_tablet_manager.cc:1905] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.288755 20597 raft_consensus.cc:482] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:18.288946 20597 raft_consensus.cc:504] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.290560 20597 leader_election.cc:288] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.295892 20632 ts_tablet_manager.cc:1918] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:18.296716 20632 log.cc:1196] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/729c191696b044ec91f7a8ccfaa68aff
I20260812 06:51:18.297323 20632 ts_tablet_manager.cc:1939] T 729c191696b044ec91f7a8ccfaa68aff P a3823bd04326435d8a3f0a463d0d2385: Deleting consensus metadata
I20260812 06:51:18.298091 20589 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: No bootstrap required, opened a new log
I20260812 06:51:18.298241 20589 ts_tablet_manager.cc:1397] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.007s
I20260812 06:51:18.299072 20589 raft_consensus.cc:348] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.299214 19640 catalog_manager.cc:4841] TS a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169): tablet 729c191696b044ec91f7a8ccfaa68aff (table test-table [id=f3f50e5369624ed5a12f8eddca43fb02]) successfully deleted
I20260812 06:51:18.299283 20589 raft_consensus.cc:374] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.299302 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.291029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:43609 (local address 127.17.69.67:54371)
0812 06:51:18.291230 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.291251 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.291366 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.292320 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.292326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.292353 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.292559 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.292570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.294065 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.294071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.296810 (+ 2739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.296826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.298583 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.298589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.298774 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.298810 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.298901 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.299029 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:51:18.299365 20589 raft_consensus.cc:729] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3823bd04326435d8a3f0a463d0d2385, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.299572 20589 consensus_queue.cc:260] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.299985 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.291644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:54371 (local address 127.17.69.65:43609)
0812 06:51:18.291826 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.291832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.291851 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:51:18.291912 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.291918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.292011 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.292116 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.292762 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.293879 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.296996 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.298217 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.298418 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.298497 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.299547 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.299680 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:51:18.300135 20589 ts_tablet_manager.cc:1428] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 06:51:18.301062 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ead805e8fb4b938910457625e4480d" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1e4002303364188ab1eba00ee335317" is_pre_election: true
W20260812 06:51:18.302248 20348 leader_election.cc:341] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:51:18.303454 20631 ts_tablet_manager.cc:1905] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.307567 20587 tablet_bootstrap.cc:492] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: No bootstrap required, opened a new log
I20260812 06:51:18.307785 20587 ts_tablet_manager.cc:1397] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.004s
I20260812 06:51:18.309175 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.291410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:44169 (local address 127.17.69.67:53387)
0812 06:51:18.300363 (+ 8953us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.300381 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.302728 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.303248 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.303252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.303276 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:18.303446 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.303457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.304870 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.304875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.306686 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.306701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.308431 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.308437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.308628 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.308661 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.308755 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.308878 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8850,"threads_started":1}
I20260812 06:51:18.309733 20587 raft_consensus.cc:348] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.310042 20587 raft_consensus.cc:374] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.310226 20587 raft_consensus.cc:729] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e4002303364188ab1eba00ee335317, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.310609 20587 consensus_queue.cc:260] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.310885 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.291871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:53387 (local address 127.17.69.66:44169)
0812 06:51:18.292013 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.292019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.300444 (+ 8425us) server_negotiation.cc:407] Connection header received
0812 06:51:18.302896 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.302903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.302994 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.303054 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:18.303599 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.304662 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.306876 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.308057 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.308247 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.308317 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.309409 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.309540 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:51:18.311587 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ead805e8fb4b938910457625e4480d" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385" is_pre_election: true
I20260812 06:51:18.311806 20213 raft_consensus.cc:2454] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 0.
I20260812 06:51:18.312325 20347 leader_election.cc:302] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, a3823bd04326435d8a3f0a463d0d2385; no voters: b1e4002303364188ab1eba00ee335317
I20260812 06:51:18.312673 20631 ts_tablet_manager.cc:1918] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:18.312839 20631 log.cc:1196] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/729c191696b044ec91f7a8ccfaa68aff
I20260812 06:51:18.311367 20587 ts_tablet_manager.cc:1428] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:51:18.313007 20597 raft_consensus.cc:2781] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:18.313118 20597 raft_consensus.cc:482] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:18.313200 20597 raft_consensus.cc:3037] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.313414 20631 ts_tablet_manager.cc:1939] T 729c191696b044ec91f7a8ccfaa68aff P b1e4002303364188ab1eba00ee335317: Deleting consensus metadata
I20260812 06:51:18.314992 19641 catalog_manager.cc:4841] TS b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609): tablet 729c191696b044ec91f7a8ccfaa68aff (table test-table [id=f3f50e5369624ed5a12f8eddca43fb02]) successfully deleted
I20260812 06:51:18.315827 20597 raft_consensus.cc:504] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.316342 20597 leader_election.cc:288] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 election: Requested vote from peers b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.316732 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ead805e8fb4b938910457625e4480d" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1e4002303364188ab1eba00ee335317"
I20260812 06:51:18.316852 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ead805e8fb4b938910457625e4480d" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385"
I20260812 06:51:18.316964 19982 raft_consensus.cc:3037] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.317013 20213 raft_consensus.cc:3037] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.319075 20213 raft_consensus.cc:2454] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 1.
I20260812 06:51:18.319479 20347 leader_election.cc:302] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, a3823bd04326435d8a3f0a463d0d2385; no voters:
I20260812 06:51:18.319723 20597 raft_consensus.cc:2781] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:18.320010 19982 raft_consensus.cc:2454] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 1.
I20260812 06:51:18.320190 20597 raft_consensus.cc:686] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 1 LEADER]: Becoming Leader. State: Replica: 3d68f6f863e647349aa6c201da4c5d5b, State: Running, Role: LEADER
I20260812 06:51:18.320647 20597 consensus_queue.cc:237] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.323550 19663 catalog_manager.cc:5495] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67). New cstate: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } health_report { overall_health: HEALTHY } } }
Aug 12 06:51:18 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517476, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:18.370805 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.356535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.17.69.126:34121)
0812 06:51:18.356713 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.356729 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.357282 (+ 553us) server_negotiation.cc:407] Connection header received
0812 06:51:18.357448 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.357453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.357550 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.357659 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.358780 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.359901 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.361092 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.361749 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.361884 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.365376 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.365381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.365384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.365416 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.367827 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.368809 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.368813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.368815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.368938 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.369307 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.369310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.369312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.369696 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.369895 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.370389 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.370557 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:51:18.371193 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.356611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34121 (local address 127.0.0.1:45024)
0812 06:51:18.357182 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.357200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.357316 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.357687 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.357691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.358329 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:18.358634 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.358644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.360015 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.360019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.360951 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.360961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.361914 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.361918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.362038 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.362666 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:18.362683 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:18.365184 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:18.367958 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:18.367971 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:18.367980 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:18.368647 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:18.369044 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:18.369050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:18.369054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:18.369217 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:18.369888 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:18.369897 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:18.370104 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.370286 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.370729 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":114,"thread_start_us":315,"threads_started":1}
I20260812 06:51:18.376667 19663 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:45024:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:51:18.378201 19663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:18.385243 19931 tablet_service.cc:1467] Processing CreateTablet for tablet 64aa2c37a6194afc8b83960c742a381b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bdb1e1bdeb64bb69ded67e96133fe86]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:18.385596 20435 tablet_service.cc:1467] Processing CreateTablet for tablet 64aa2c37a6194afc8b83960c742a381b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bdb1e1bdeb64bb69ded67e96133fe86]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:18.385571 20178 tablet_service.cc:1467] Processing CreateTablet for tablet 64aa2c37a6194afc8b83960c742a381b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bdb1e1bdeb64bb69ded67e96133fe86]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:18.385859 19931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64aa2c37a6194afc8b83960c742a381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.386075 20435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64aa2c37a6194afc8b83960c742a381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.386132 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64aa2c37a6194afc8b83960c742a381b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:18.393061 20587 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: Bootstrap starting.
I20260812 06:51:18.393074 20591 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: Bootstrap starting.
I20260812 06:51:18.393117 20589 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: Bootstrap starting.
I20260812 06:51:18.394676 20591 tablet_bootstrap.cc:654] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.394763 20589 tablet_bootstrap.cc:654] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.394811 20587 tablet_bootstrap.cc:654] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:18.405068 20591 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: No bootstrap required, opened a new log
I20260812 06:51:18.405306 20591 ts_tablet_manager.cc:1397] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20260812 06:51:18.406129 20591 raft_consensus.cc:348] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.406468 20591 raft_consensus.cc:374] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.406576 20591 raft_consensus.cc:729] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d68f6f863e647349aa6c201da4c5d5b, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.406858 20591 consensus_queue.cc:260] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.407413 20597 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.407723 20597 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.408272 20591 ts_tablet_manager.cc:1428] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:18.416277 20587 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: No bootstrap required, opened a new log
I20260812 06:51:18.416460 20587 ts_tablet_manager.cc:1397] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.004s
I20260812 06:51:18.416951 20589 tablet_bootstrap.cc:492] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: No bootstrap required, opened a new log
I20260812 06:51:18.417130 20587 raft_consensus.cc:348] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.417376 20587 raft_consensus.cc:374] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.417469 20587 raft_consensus.cc:729] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e4002303364188ab1eba00ee335317, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.417598 20589 ts_tablet_manager.cc:1397] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.012s
I20260812 06:51:18.417929 20587 consensus_queue.cc:260] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.418443 20625 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.418637 20589 raft_consensus.cc:348] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.418857 20625 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.418886 20589 raft_consensus.cc:374] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:18.418957 20589 raft_consensus.cc:729] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3823bd04326435d8a3f0a463d0d2385, State: Initialized, Role: FOLLOWER
I20260812 06:51:18.419217 20589 consensus_queue.cc:260] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.419909 20587 ts_tablet_manager.cc:1428] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 06:51:18.419713 20599 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.420075 20599 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.421229 20589 ts_tablet_manager.cc:1428] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:18.479346 20597 raft_consensus.cc:482] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:18.479522 20597 raft_consensus.cc:504] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.480110 20597 leader_election.cc:288] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.480630 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64aa2c37a6194afc8b83960c742a381b" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385" is_pre_election: true
I20260812 06:51:18.480674 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64aa2c37a6194afc8b83960c742a381b" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1e4002303364188ab1eba00ee335317" is_pre_election: true
I20260812 06:51:18.480916 19982 raft_consensus.cc:2454] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 0.
I20260812 06:51:18.480923 20213 raft_consensus.cc:2454] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 0.
I20260812 06:51:18.481361 20348 leader_election.cc:302] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, b1e4002303364188ab1eba00ee335317; no voters:
I20260812 06:51:18.481678 20597 raft_consensus.cc:2781] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:18.481796 20597 raft_consensus.cc:482] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:18.481904 20597 raft_consensus.cc:3037] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.484881 20597 raft_consensus.cc:504] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.485595 20597 leader_election.cc:288] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 election: Requested vote from peers b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609), a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169)
I20260812 06:51:18.486038 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64aa2c37a6194afc8b83960c742a381b" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1e4002303364188ab1eba00ee335317"
I20260812 06:51:18.486020 20213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64aa2c37a6194afc8b83960c742a381b" candidate_uuid: "3d68f6f863e647349aa6c201da4c5d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3823bd04326435d8a3f0a463d0d2385"
I20260812 06:51:18.486243 19982 raft_consensus.cc:3037] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.486240 20213 raft_consensus.cc:3037] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:18.488402 19982 raft_consensus.cc:2454] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 1.
I20260812 06:51:18.488792 20348 leader_election.cc:302] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d68f6f863e647349aa6c201da4c5d5b, b1e4002303364188ab1eba00ee335317; no voters:
I20260812 06:51:18.489027 20597 raft_consensus.cc:2781] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:18.489153 20213 raft_consensus.cc:2454] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d68f6f863e647349aa6c201da4c5d5b in term 1.
I20260812 06:51:18.489231 20597 raft_consensus.cc:686] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [term 1 LEADER]: Becoming Leader. State: Replica: 3d68f6f863e647349aa6c201da4c5d5b, State: Running, Role: LEADER
W20260812 06:51:18.489590 20056 tablet.cc:2369] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:18.490003 20597 consensus_queue.cc:237] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } }
I20260812 06:51:18.491707 20642 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: TxnStatusTablet state changed. Reason: New leader 3d68f6f863e647349aa6c201da4c5d5b. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.492013 20642 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:18.492681 20660 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:51:18.492928 20660 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:51:18.493121 19662 catalog_manager.cc:5495] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67). New cstate: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } health_report { overall_health: HEALTHY } } }
I20260812 06:51:18.533679 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.526037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.17.69.67:42359)
0812 06:51:18.526384 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.526391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.526417 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:51:18.526686 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.526692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.526818 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.526977 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:18.527375 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.528597 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.530678 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.531462 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.531590 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.532185 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.532963 (+ 778us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.533101 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.533262 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.533360 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260812 06:51:18.533699 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.525929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:42359 (local address 127.0.0.1:37090)
0812 06:51:18.526290 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.526311 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.526474 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.527001 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.527006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.527019 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:18.527221 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.527235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.528759 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.528765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.530524 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.530541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.531639 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.531646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.531873 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.531953 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.533083 (+ 1130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.533114 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.533215 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.533389 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260812 06:51:18.539534 20213 raft_consensus.cc:1261] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Refusing update from remote peer 3d68f6f863e647349aa6c201da4c5d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:18.539722 19982 raft_consensus.cc:1261] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Refusing update from remote peer 3d68f6f863e647349aa6c201da4c5d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:18.540367 20597 consensus_queue.cc:1035] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.540719 20642 consensus_queue.cc:1035] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.555425 20625 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: TxnStatusTablet state changed. Reason: New leader 3d68f6f863e647349aa6c201da4c5d5b. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.555757 20625 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.556242 20599 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: TxnStatusTablet state changed. Reason: New leader 3d68f6f863e647349aa6c201da4c5d5b. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.556535 20599 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.559798 20597 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.560105 20597 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:18.560330 20617 mvcc.cc:205] Tried to move back new op lower bound from 7317575592089415680 to 7317575591897354240. Current Snapshot: MvccSnapshot[applied={T|T < 7317575592089415680}]
I20260812 06:51:18.560493 20625 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.568099 20642 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.568384 20642 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P 3d68f6f863e647349aa6c201da4c5d5b: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:18.569562 20599 tablet_replica.cc:439] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d68f6f863e647349aa6c201da4c5d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 } } peers { permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 } } peers { permanent_uuid: "3d68f6f863e647349aa6c201da4c5d5b" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42359 } } }
I20260812 06:51:18.569830 20599 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P a3823bd04326435d8a3f0a463d0d2385: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.572933 20625 tablet_replica.cc:442] T 64aa2c37a6194afc8b83960c742a381b P b1e4002303364188ab1eba00ee335317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:18.582947 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.576139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.17.69.67:42359)
0812 06:51:18.576371 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.576378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.576396 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:51:18.576457 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.576462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.576537 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.576646 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:18.577271 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.578480 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.580394 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.581060 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.581226 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.581864 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.581990 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.582075 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.582570 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.582649 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:51:18.618867 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.595321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:38483 (local address 127.17.69.67:42359)
0812 06:51:18.595476 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.595481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.612026 (+ 16545us) server_negotiation.cc:407] Connection header received
0812 06:51:18.612297 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.612302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.612379 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.612433 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:18.613336 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.614573 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.616686 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.617357 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.617448 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.618056 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.618207 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.618279 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.618477 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.618553 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:51:18.619778 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.595355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:42359 (local address 127.17.69.67:38483)
0812 06:51:18.611917 (+ 16562us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:18.611941 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:18.612141 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:18.612627 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:18.612632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:18.612645 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:18.613167 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.613181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.614734 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.614738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.616539 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:18.616555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.617635 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.617641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:18.617799 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.617880 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:18.618365 (+ 485us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:18.618374 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:51:18.619245 (+ 871us) client_negotiation.cc:240] Negotiation successful
0812 06:51:18.619383 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16381,"thread_start_us":15179,"threads_started":1}
I20260812 06:51:18.633944 19982 raft_consensus.cc:1261] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Refusing update from remote peer 3d68f6f863e647349aa6c201da4c5d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:18.633946 20212 raft_consensus.cc:1261] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Refusing update from remote peer 3d68f6f863e647349aa6c201da4c5d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:18.634665 20597 consensus_queue.cc:1035] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3823bd04326435d8a3f0a463d0d2385" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.635185 20597 consensus_queue.cc:1035] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1e4002303364188ab1eba00ee335317" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:18.752585 19662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:18.752884 19662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:18.756747 19662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 451af43be29543ecbdf5fb8b783970d9: Sending DeleteTablet for 3 replicas of tablet 81ead805e8fb4b938910457625e4480d
I20260812 06:51:18.757480 19931 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ead805e8fb4b938910457625e4480d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39912
I20260812 06:51:18.757774 20631 tablet_replica.cc:330] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: stopping tablet replica
I20260812 06:51:18.757987 20631 raft_consensus.cc:2229] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:18.758179 20631 raft_consensus.cc:2258] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.758705 20178 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ead805e8fb4b938910457625e4480d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:43386
I20260812 06:51:18.758998 20632 tablet_replica.cc:330] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: stopping tablet replica
I20260812 06:51:18.759196 20632 raft_consensus.cc:2229] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:18.759464 20632 raft_consensus.cc:2258] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.759786 20437 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ead805e8fb4b938910457625e4480d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:18 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:37082
I20260812 06:51:18.760044 20634 tablet_replica.cc:330] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: stopping tablet replica
I20260812 06:51:18.760237 20634 raft_consensus.cc:2229] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:18.760716 20634 raft_consensus.cc:2258] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:18.762130 20634 ts_tablet_manager.cc:1905] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.768911 20632 ts_tablet_manager.cc:1905] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.774798 20631 ts_tablet_manager.cc:1905] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:18.776005 20632 ts_tablet_manager.cc:1918] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:18.776161 20632 log.cc:1196] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/81ead805e8fb4b938910457625e4480d
I20260812 06:51:18.776648 20632 ts_tablet_manager.cc:1939] T 81ead805e8fb4b938910457625e4480d P a3823bd04326435d8a3f0a463d0d2385: Deleting consensus metadata
I20260812 06:51:18.777885 19640 catalog_manager.cc:4841] TS a3823bd04326435d8a3f0a463d0d2385 (127.17.69.66:44169): tablet 81ead805e8fb4b938910457625e4480d (table test-table [id=344e11da7a5f4247b292c02043924b0c]) successfully deleted
I20260812 06:51:18.780259 20634 ts_tablet_manager.cc:1918] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:18.781883 20631 ts_tablet_manager.cc:1918] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:18.782089 20631 log.cc:1196] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/81ead805e8fb4b938910457625e4480d
I20260812 06:51:18.782112 20634 log.cc:1196] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/81ead805e8fb4b938910457625e4480d
I20260812 06:51:18.782658 20634 ts_tablet_manager.cc:1939] T 81ead805e8fb4b938910457625e4480d P 3d68f6f863e647349aa6c201da4c5d5b: Deleting consensus metadata
I20260812 06:51:18.782670 20631 ts_tablet_manager.cc:1939] T 81ead805e8fb4b938910457625e4480d P b1e4002303364188ab1eba00ee335317: Deleting consensus metadata
I20260812 06:51:18.783711 19641 catalog_manager.cc:4841] TS b1e4002303364188ab1eba00ee335317 (127.17.69.65:43609): tablet 81ead805e8fb4b938910457625e4480d (table test-table [id=344e11da7a5f4247b292c02043924b0c]) successfully deleted
I20260812 06:51:18.784308 19638 catalog_manager.cc:4841] TS 3d68f6f863e647349aa6c201da4c5d5b (127.17.69.67:42359): tablet 81ead805e8fb4b938910457625e4480d (table test-table [id=344e11da7a5f4247b292c02043924b0c]) successfully deleted
Aug 12 06:51:18 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:18.792496 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.775599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.17.69.65:43609)
0812 06:51:18.775789 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.775795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.776590 (+ 795us) server_negotiation.cc:407] Connection header received
0812 06:51:18.776851 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.776857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.776957 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.777030 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.779160 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.780322 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.782180 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.782812 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.782907 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.787085 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.787097 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.787111 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.787147 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.789887 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.790557 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.790564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.790567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.790700 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.791065 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.791069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.791071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.791418 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.791692 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.792001 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.792220 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 06:51:18.793223 20047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39930
I20260812 06:51:18.808738 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.796714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.17.69.126:34121)
0812 06:51:18.796942 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.796948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.797365 (+ 417us) server_negotiation.cc:407] Connection header received
0812 06:51:18.797503 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.797509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.797583 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.797665 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.798951 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.799664 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.800855 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.801309 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.801421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.803570 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.803575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.803579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.803613 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.806199 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.806858 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.806864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.806868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.806994 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.807336 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.807341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.807345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.807647 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.807904 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.808208 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.808441 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 06:51:18.809485 19662 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45026
I20260812 06:51:18.811623 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19808
I20260812 06:51:18.825060 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.812711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.17.69.65:43609)
0812 06:51:18.812887 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.812893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.812911 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:51:18.813127 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.813132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.813212 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.813322 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.814318 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.815427 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.816659 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.817250 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.817388 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.819418 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.819425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.819429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.819469 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.822662 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.823220 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.823226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.823229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.823396 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.823697 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.823702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.823705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.824037 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.824289 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.824460 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.824693 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 06:51:18.825659 20047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39934
W20260812 06:51:18.826187 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19808: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:18.826426 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19808
I20260812 06:51:18.857528 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20060
Aug 12 06:51:18 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:18.872735 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.858389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.17.69.66:44169)
0812 06:51:18.858841 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.858845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.858860 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:51:18.858914 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.858918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.858984 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.859079 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.860230 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.860926 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.862148 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.862611 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.862731 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.866821 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.866835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.866848 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.866881 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.869875 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.870876 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.870882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.870885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.871007 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.871421 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.871424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.871425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.871748 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.871960 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.872296 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.872452 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":229,"threads_started":1}
W20260812 06:51:18.873520 20287 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43388
W20260812 06:51:18.874179 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20060: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:18.874395 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20060
I20260812 06:51:18.904036 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20307
Aug 12 06:51:18 dist-test-slave-6zbq krb5kdc[19574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:18.919680 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:18.904798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.17.69.67:42359)
0812 06:51:18.904998 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:18.905004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:18.905090 (+ 86us) server_negotiation.cc:407] Connection header received
0812 06:51:18.905280 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:18.905284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:18.905376 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:18.905477 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:18.906797 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.907586 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.908904 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:18.909372 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:18.909454 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:18.913941 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:18.913953 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:18.913970 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:18.914008 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:18.916868 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.917492 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.917500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.917502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.917627 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:18.917990 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:18.917994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:18.917996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:18.918555 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:18.918891 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:18.919149 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 06:51:18.919376 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 06:51:18.920354 20546 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37108
W20260812 06:51:18.920985 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20307: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:18.921151 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20307
I20260812 06:51:18.948062 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19603
I20260812 06:51:18.949162 19748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:19.064165 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19603
2026-08-12T06:51:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4465 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:19 dist-test-slave-6zbq krb5kdc[20694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:19 dist-test-slave-6zbq krb5kdc[20694](info): set up 2 sockets
Aug 12 06:51:19 dist-test-slave-6zbq krb5kdc[20694](info): commencing operation
krb5kdc: starting...
W20260812 06:51:20.282684 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:20 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:20.463440 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:33893
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:33893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:20.778409 20719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:20.778684 20719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:20.778779 20719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:20.785761 20719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:20.785853 20719 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:51:20.785900 20719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:20.785959 20719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:20.786006 20719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:20.786064 20719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:20.795408 20719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:33893
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:20.796540 20719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:20.797801 20719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:20.808992 20727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:20.809067 20728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:20.809569 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:20.809520 20719 server_base.cc:1034] running on GCE node
I20260812 06:51:20.810439 20719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:20.811743 20719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:20.812978 20719 hybrid_clock.cc:648] HybridClock initialized: now 1786517480812912 us; error 34 us; skew 500 ppm
I20260812 06:51:20.813298 20719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:20 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517480, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:20.816244 20719 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:20.817533 20719 webserver.cc:460] Webserver started at http://127.17.69.126:40025/ using document root <none> and password file <none>
I20260812 06:51:20.818172 20719 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:20.818256 20719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:20.818549 20719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:20.821342 20719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "32135cbdf20347abb1ae4554a42f2fb5"
format_stamp: "Formatted at 2026-08-12 06:51:20 on dist-test-slave-6zbq"
server_key: "912e16e214539fcb8273ea22319393c3"
server_key_iv: "13dff729fd9abbcd157d00690080d597"
server_key_version: "encryptionkey@0"
I20260812 06:51:20.822046 20719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "32135cbdf20347abb1ae4554a42f2fb5"
format_stamp: "Formatted at 2026-08-12 06:51:20 on dist-test-slave-6zbq"
server_key: "912e16e214539fcb8273ea22319393c3"
server_key_iv: "13dff729fd9abbcd157d00690080d597"
server_key_version: "encryptionkey@0"
I20260812 06:51:20.826773 20719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:20.829872 20752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:20.831056 20719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:20.831202 20719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "32135cbdf20347abb1ae4554a42f2fb5"
format_stamp: "Formatted at 2026-08-12 06:51:20 on dist-test-slave-6zbq"
server_key: "912e16e214539fcb8273ea22319393c3"
server_key_iv: "13dff729fd9abbcd157d00690080d597"
server_key_version: "encryptionkey@0"
I20260812 06:51:20.831368 20719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:20.857185 20719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:20.860844 20719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:20.861047 20719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:20.879521 20719 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:33893
I20260812 06:51:20.879559 20873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:33893 every 8 connection(s)
I20260812 06:51:20.880672 20719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:20.884038 20875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:20.888985 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20719
I20260812 06:51:20.889176 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:20.889799 17685 external_mini_cluster.cc:1428] Setting key bb043cc83e79b5e1a859c0081bb9b9e9
I20260812 06:51:20.893990 20875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Bootstrap starting.
I20260812 06:51:20.896755 20875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:20.897691 20875 log.cc:824] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:20.900252 20875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: No bootstrap required, opened a new log
Aug 12 06:51:20 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:20.909262 20875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } }
I20260812 06:51:20.909682 20875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:20.909801 20875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32135cbdf20347abb1ae4554a42f2fb5, State: Initialized, Role: FOLLOWER
I20260812 06:51:20.910511 20875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } }
I20260812 06:51:20.910782 20875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:20.910933 20875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:20.911118 20875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:20.911752 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:20.891685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.17.69.126:33893)
0812 06:51:20.892466 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:20.892477 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:20.892513 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:51:20.894144 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:20.894175 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:20.894574 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:20.894903 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:20.896343 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:20.897642 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:20.898898 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:20.899527 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:20.899809 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:20.903599 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:20.903609 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:20.903625 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:20.903670 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:20.907376 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:20.908047 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:20.908055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:20.908067 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:20.908273 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:20.908630 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:20.908635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:20.908638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:20.909294 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:20.909585 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:20.909998 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 06:51:20.910421 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":348,"threads_started":1}
I20260812 06:51:20.913868 20875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } }
I20260812 06:51:20.914412 20875 leader_election.cc:302] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32135cbdf20347abb1ae4554a42f2fb5; no voters:
I20260812 06:51:20.914940 20875 leader_election.cc:288] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:20.915038 20888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:20.915680 20888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [term 1 LEADER]: Becoming Leader. State: Replica: 32135cbdf20347abb1ae4554a42f2fb5, State: Running, Role: LEADER
I20260812 06:51:20.916153 20888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } }
I20260812 06:51:20.916706 20875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:20.918679 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32135cbdf20347abb1ae4554a42f2fb5. Latest consensus state: current_term: 1 leader_uuid: "32135cbdf20347abb1ae4554a42f2fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } } }
I20260812 06:51:20.919070 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:20.919530 20890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32135cbdf20347abb1ae4554a42f2fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32135cbdf20347abb1ae4554a42f2fb5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 33893 } } }
I20260812 06:51:20.919855 20890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:20.921913 20900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:20.926898 20900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:20.936123 20900 catalog_manager.cc:1347] Generated new cluster ID: c796b6a333464f398b61155f7292e12f
I20260812 06:51:20.936237 20900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:20.953425 20900 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:20.955323 20900 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:20.976156 20900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Generated new TSK 0
I20260812 06:51:20.976902 20900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:21.029155 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:33893
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:21.353413 20928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:21.353659 20928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:21.353798 20928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:21.360754 20928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:21.360841 20928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:21.360960 20928 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:51:21.361021 20928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:21.361135 20928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:21.370262 20928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:21.371351 20928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:21.372565 20928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:21.384265 20935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:21.384233 20936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:21.384316 20942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:21.385109 20928 server_base.cc:1034] running on GCE node
I20260812 06:51:21.385645 20928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:21.386467 20928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:21.387650 20928 hybrid_clock.cc:648] HybridClock initialized: now 1786517481387600 us; error 48 us; skew 500 ppm
I20260812 06:51:21.387908 20928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:21 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:21.390775 20928 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:21.392134 20928 webserver.cc:460] Webserver started at http://127.17.69.65:36623/ using document root <none> and password file <none>
I20260812 06:51:21.392686 20928 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:21.392789 20928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:21.393030 20928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:21.395675 20928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "c3eec83786cc48ef864cd40c910c7881"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "87373df00a05fc4cd44e11253a2a8c64"
server_key_iv: "52bcc88b84afa552c6cbdbd57dabd10c"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.396346 20928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "c3eec83786cc48ef864cd40c910c7881"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "87373df00a05fc4cd44e11253a2a8c64"
server_key_iv: "52bcc88b84afa552c6cbdbd57dabd10c"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.401154 20928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:51:21.404279 20953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:21.405505 20928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:51:21.405740 20928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "c3eec83786cc48ef864cd40c910c7881"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "87373df00a05fc4cd44e11253a2a8c64"
server_key_iv: "52bcc88b84afa552c6cbdbd57dabd10c"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.405932 20928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:21.427457 20928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:21.431216 20928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:21.431450 20928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:21.433799 20928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:21.433923 20928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:21.434099 20928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:21.434194 20928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:21 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:21.455931 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.434222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:36911 (local address 127.17.69.126:33893)
0812 06:51:21.434720 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:21.434727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:21.435963 (+ 1236us) server_negotiation.cc:407] Connection header received
0812 06:51:21.438710 (+ 2747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:21.438714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:21.438784 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:21.438884 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:21.441135 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.441882 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.444681 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.445323 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.445462 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.449409 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:21.449416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:21.449420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:21.449458 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:21.452517 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.453467 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.453473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.453476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.453636 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.454154 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.454160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.454164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.454518 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:21.454657 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:21.455356 (+ 699us) server_negotiation.cc:299] Negotiation successful
0812 06:51:21.455576 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260812 06:51:21.456795 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.434740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.65:36911)
0812 06:51:21.435790 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:21.435842 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:21.438416 (+ 2574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:21.439153 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:21.439163 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:21.440032 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:21.440917 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.440952 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.442026 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.442033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.444478 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.444493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.445519 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.445525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.445848 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.446461 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:21.446482 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:21.449146 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:21.452669 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.452677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.452694 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.453314 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.453789 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.453797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.453801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.454021 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.454653 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:21.454661 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:21.454971 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 06:51:21.455195 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 06:51:21.455665 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":389,"threads_started":1}
I20260812 06:51:21.475824 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.462501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:45019 (local address 127.17.69.126:33893)
0812 06:51:21.462690 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:21.462696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:21.463556 (+ 860us) server_negotiation.cc:407] Connection header received
0812 06:51:21.463706 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:21.463710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:21.463775 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:21.463849 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:21.464973 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.465650 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.467584 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.467986 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.468086 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.470379 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:21.470387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:21.470392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:21.470429 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:21.472958 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.473835 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.473841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.473844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.473977 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.474335 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.474364 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.474368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.474629 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:21.474734 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:21.475255 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 06:51:21.475495 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:51:21.476004 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.462444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.65:45019)
0812 06:51:21.463445 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:21.463471 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:21.463606 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:21.463887 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:21.463892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:21.464461 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:21.464853 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.464865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.465793 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.465799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.467429 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.467445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.468156 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.468163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.468354 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.468917 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:21.468937 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:21.470163 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:21.473075 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.473081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.473086 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.473702 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.474081 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.474087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.474091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.474247 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.474721 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:21.474726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:21.474956 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 06:51:21.475128 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 06:51:21.475481 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":628,"threads_started":1}
I20260812 06:51:21.486332 20928 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:33541
I20260812 06:51:21.486332 21160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:33541 every 8 connection(s)
I20260812 06:51:21.487464 20928 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:21.489210 21161 heartbeater.cc:347] Connected to a master server at 127.17.69.126:33893
I20260812 06:51:21.489413 21161 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:21.489982 21161 heartbeater.cc:511] Master 127.17.69.126:33893 requested a full tablet report, sending...
I20260812 06:51:21.491739 20787 ts_manager.cc:194] Registered new tserver with Master: c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541)
I20260812 06:51:21.493090 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:36911
I20260812 06:51:21.496836 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20928
I20260812 06:51:21.496992 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:21.497601 17685 external_mini_cluster.cc:1428] Setting key ad1d17da202fd666fe643b0f1000a64e
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:21.539353 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:33893
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:21.851387 21166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:21.851655 21166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:21.851805 21166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:21.858733 21166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:21.858847 21166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:21.858966 21166 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:51:21.859050 21166 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:21.859153 21166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:21.868404 21166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:21.869547 21166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:21.870788 21166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:21.882627 21176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:21.882726 21182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:21.882992 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:21.883764 21166 server_base.cc:1034] running on GCE node
I20260812 06:51:21.884413 21166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:21.885397 21166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:21.886646 21166 hybrid_clock.cc:648] HybridClock initialized: now 1786517481886570 us; error 47 us; skew 500 ppm
I20260812 06:51:21.886971 21166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:21 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:21.890681 21166 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:21.892223 21166 webserver.cc:460] Webserver started at http://127.17.69.66:41065/ using document root <none> and password file <none>
I20260812 06:51:21.892834 21166 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:21.892947 21166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:21.893234 21166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:21.896174 21166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "28c6b135b04d43bdb3f2e87342575c47"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "c1e29461e2271b99289528dc525137df"
server_key_iv: "f93f135cdbab161b5fe36fda48f6363e"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.896921 21166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "28c6b135b04d43bdb3f2e87342575c47"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "c1e29461e2271b99289528dc525137df"
server_key_iv: "f93f135cdbab161b5fe36fda48f6363e"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.901813 21166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 06:51:21.905117 21197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:21.906276 21166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:21.906467 21166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "28c6b135b04d43bdb3f2e87342575c47"
format_stamp: "Formatted at 2026-08-12 06:51:21 on dist-test-slave-6zbq"
server_key: "c1e29461e2271b99289528dc525137df"
server_key_iv: "f93f135cdbab161b5fe36fda48f6363e"
server_key_version: "encryptionkey@0"
I20260812 06:51:21.906646 21166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:21.932578 21166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:21.936537 21166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:21.936800 21166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:21.939329 21166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:21.939433 21166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:21.939599 21166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:21.939723 21166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:21 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517481, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:21.962235 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.939707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:50577 (local address 127.17.69.126:33893)
0812 06:51:21.939918 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:21.939934 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:21.941324 (+ 1390us) server_negotiation.cc:407] Connection header received
0812 06:51:21.943547 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:21.943553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:21.943643 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:21.943764 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:21.945792 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.946830 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.949224 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.949913 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.950047 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.955022 (+ 4975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:21.955029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:21.955035 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:21.955075 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:21.958328 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.959368 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.959374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.959377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.959575 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.960119 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.960124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.960128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.960558 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:21.960734 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:21.961593 (+ 859us) server_negotiation.cc:299] Negotiation successful
0812 06:51:21.961851 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:51:21.963142 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.940244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.66:50577)
0812 06:51:21.941124 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:21.941185 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:21.943209 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:21.944029 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:21.944042 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:21.944751 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:21.945604 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.945635 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.947015 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.947019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.949079 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.949095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.950452 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.950463 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.950758 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.951541 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:21.951575 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:21.954808 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:21.958480 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.958489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.958507 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.959185 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.959726 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.959730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.959734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.960005 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.960706 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:21.960717 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:21.961169 (+ 452us) client_negotiation.cc:769] Sending connection context
0812 06:51:21.961481 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 06:51:21.961914 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":364,"threads_started":1}
I20260812 06:51:21.982108 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.969654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:36339 (local address 127.17.69.126:33893)
0812 06:51:21.969839 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:21.969845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:21.969916 (+ 71us) server_negotiation.cc:407] Connection header received
0812 06:51:21.970056 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:21.970062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:21.970137 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:21.970242 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:21.971451 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.972393 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.973931 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.974542 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.974673 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.976879 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:21.976885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:21.976890 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:21.976925 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:21.979585 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.980184 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.980189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.980193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.980359 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:21.980746 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:21.980752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:21.980755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:21.981091 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:21.981244 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:21.981734 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 06:51:21.981894 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:51:21.982523 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:21.969224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.66:36339)
0812 06:51:21.969843 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:21.969862 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:21.969963 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:21.970249 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:21.970252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:21.970839 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:21.971245 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.971260 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.972521 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.972529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.973777 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:21.973790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:21.974763 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:21.974774 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:21.974990 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:21.975641 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:21.975676 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:21.976696 (+ 1020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:21.979708 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.979713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.979717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.980074 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.980492 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:21.980497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:21.980501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:21.980641 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:21.981230 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:21.981239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:21.981491 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 06:51:21.981664 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 06:51:21.982003 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":327,"threads_started":1}
I20260812 06:51:21.984866 21166 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:46493
I20260812 06:51:21.984898 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:46493 every 8 connection(s)
I20260812 06:51:21.986104 21166 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:21.987916 21394 heartbeater.cc:347] Connected to a master server at 127.17.69.126:33893
I20260812 06:51:21.988119 21394 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:21.988689 21394 heartbeater.cc:511] Master 127.17.69.126:33893 requested a full tablet report, sending...
I20260812 06:51:21.989910 20787 ts_manager.cc:194] Registered new tserver with Master: 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493)
I20260812 06:51:21.990706 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:50577
I20260812 06:51:21.995850 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21166
I20260812 06:51:21.996002 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:21.996440 17685 external_mini_cluster.cc:1428] Setting key ebc8be4bc80d31b302bf02f6787b1df5
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:22.039294 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:33893
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:22.352313 21401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:22.352574 21401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:22.352721 21401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:22.359786 21401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:22.359889 21401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:22.360001 21401 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 06:51:22.360066 21401 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:51:22.360165 21401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:22.369434 21401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:22.370602 21401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:22.371814 21401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:22.383948 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:22.384027 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:22.383980 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:22.384632 21401 server_base.cc:1034] running on GCE node
I20260812 06:51:22.385202 21401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:22.386008 21401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:22.387202 21401 hybrid_clock.cc:648] HybridClock initialized: now 1786517482387157 us; error 35 us; skew 500 ppm
I20260812 06:51:22.387497 21401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:22 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517482, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:22.394860 21401 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:22.396283 21401 webserver.cc:460] Webserver started at http://127.17.69.67:37233/ using document root <none> and password file <none>
I20260812 06:51:22.396904 21401 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:22.397004 21401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:22.397271 21401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:22.400102 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "4833e71e4623433db68d8e7d5e6570de"
format_stamp: "Formatted at 2026-08-12 06:51:22 on dist-test-slave-6zbq"
server_key: "6f0c81667ad8d170a011f11540450b62"
server_key_iv: "758643a5bf60e5cace677033c89433ab"
server_key_version: "encryptionkey@0"
I20260812 06:51:22.400806 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "4833e71e4623433db68d8e7d5e6570de"
format_stamp: "Formatted at 2026-08-12 06:51:22 on dist-test-slave-6zbq"
server_key: "6f0c81667ad8d170a011f11540450b62"
server_key_iv: "758643a5bf60e5cace677033c89433ab"
server_key_version: "encryptionkey@0"
I20260812 06:51:22.405817 21401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 06:51:22.409246 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:22.410651 21401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:51:22.410811 21401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "4833e71e4623433db68d8e7d5e6570de"
format_stamp: "Formatted at 2026-08-12 06:51:22 on dist-test-slave-6zbq"
server_key: "6f0c81667ad8d170a011f11540450b62"
server_key_iv: "758643a5bf60e5cace677033c89433ab"
server_key_version: "encryptionkey@0"
I20260812 06:51:22.410987 21401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:22.430012 21401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:22.433970 21401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:22.434247 21401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:22.436817 21401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:22.436921 21401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:22.437040 21401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:22.437120 21401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:51:22 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517482, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:22.460604 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.437887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.67:34839)
0812 06:51:22.438780 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.438827 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.441336 (+ 2509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.442215 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.442225 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.443084 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:22.444027 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.444059 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.445242 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.445248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.447643 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.447659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.448588 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.448596 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.448882 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.449659 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:22.449690 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:22.452822 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:22.456454 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.456462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.456477 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.457031 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.457506 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.457512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.457515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.457757 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.458447 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:22.458454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:22.458841 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.459172 (+ 331us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.459618 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":383,"threads_started":1}
I20260812 06:51:22.461761 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.437223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:34839 (local address 127.17.69.126:33893)
0812 06:51:22.437444 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.437452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.438962 (+ 1510us) server_negotiation.cc:407] Connection header received
0812 06:51:22.441677 (+ 2715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.441683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.441778 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.441896 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:22.444243 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.445069 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.447810 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.448329 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.448460 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.453036 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:22.453041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:22.453045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:22.453077 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:22.456325 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.457174 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.457180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.457184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.457366 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.457879 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.457884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.457888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.458287 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:22.458472 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.461355 (+ 2883us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.461524 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:51:22.480230 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.467444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:54653 (local address 127.17.69.126:33893)
0812 06:51:22.467620 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.467626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.468167 (+ 541us) server_negotiation.cc:407] Connection header received
0812 06:51:22.468407 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.468412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.468519 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.468623 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:22.469805 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.470550 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.471849 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.472208 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.472322 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.473832 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:22.473836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:22.473840 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:22.473868 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:22.477003 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.477803 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.477808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.477810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.477929 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.478704 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.478709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.478713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.478998 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:22.479148 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.479689 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.479890 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:51:22.480414 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.467450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.17.69.67:54653)
0812 06:51:22.468065 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.468084 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.468225 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.468667 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.468672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.469298 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:22.469666 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.469678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.470695 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.470700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.471713 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.471725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.472378 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.472384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.472532 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.473012 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:22.473028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:22.473660 (+ 632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:22.477136 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.477145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.477148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.477631 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.478394 (+ 763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.478400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.478403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.478577 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.479218 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:22.479224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:22.479429 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.479569 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.479904 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
I20260812 06:51:22.484297 21401 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:40341
I20260812 06:51:22.484339 21616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:40341 every 8 connection(s)
I20260812 06:51:22.485430 21401 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:22.487458 21619 heartbeater.cc:347] Connected to a master server at 127.17.69.126:33893
I20260812 06:51:22.487668 21619 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:22.488229 21619 heartbeater.cc:511] Master 127.17.69.126:33893 requested a full tablet report, sending...
I20260812 06:51:22.489493 20787 ts_manager.cc:194] Registered new tserver with Master: 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341)
I20260812 06:51:22.490573 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:34839
I20260812 06:51:22.495489 21161 heartbeater.cc:503] Master 127.17.69.126:33893 was elected leader, sending a full tablet report...
I20260812 06:51:22.495596 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21401
I20260812 06:51:22.495749 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:22.496237 17685 external_mini_cluster.cc:1428] Setting key 4526ab4c50f2fb5a8a3bdb3f6a6f2148
I20260812 06:51:22.498837 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:51:22 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:51:22 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:22.548043 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.534902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.17.69.126:33893)
0812 06:51:22.535127 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.535133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.535355 (+ 222us) server_negotiation.cc:407] Connection header received
0812 06:51:22.535511 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.535516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.535584 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.535667 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:22.537052 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.537782 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.538995 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.539398 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.539490 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.543482 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:22.543488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:22.543491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:22.543524 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:22.545825 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.546406 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.546411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.546415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.546546 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.546842 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.546845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.546847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.547099 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:22.547288 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.547538 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.547699 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:51:22.551849 20787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:22.558459 20787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:22.589784 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.569870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46493 (local address 127.0.0.1:34142)
0812 06:51:22.581601 (+ 11731us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.581635 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.581826 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.582659 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.582665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.582690 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:22.583000 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.583009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.584437 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.584442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.586867 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.586886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.588771 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.588778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.589003 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.589060 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.589179 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.589369 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11504,"thread_start_us":11361,"threads_started":1}
I20260812 06:51:22.591363 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.570080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.17.69.66:46493)
0812 06:51:22.581200 (+ 11120us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.581210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.582019 (+ 809us) server_negotiation.cc:407] Connection header received
0812 06:51:22.582126 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.582134 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.582424 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.583174 (+ 750us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:22.583325 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.584282 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.587102 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.588520 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.590624 (+ 2104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.590843 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.590913 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.591075 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10931,"thread_start_us":10799,"threads_started":1}
I20260812 06:51:22.594456 21650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.570418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.17.69.67:40341)
0812 06:51:22.586019 (+ 15601us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.586028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.586063 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:51:22.586163 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.586171 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.586418 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.587544 (+ 1126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:22.587861 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.589538 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.592009 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.593388 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.593505 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.593786 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.593938 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.594095 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15416,"thread_start_us":16132,"threads_started":1}
I20260812 06:51:22.594722 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.570155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40341 (local address 127.0.0.1:55788)
0812 06:51:22.583623 (+ 13468us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.583642 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.583763 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.586831 (+ 3068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.586837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.586866 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:22.587334 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.587345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.589702 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.589708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.591860 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.591874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.593589 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.593595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.593764 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.593796 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.593898 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.594025 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13325,"thread_start_us":13685,"threads_started":1}
I20260812 06:51:22.595219 21278 tablet_service.cc:1467] Processing CreateTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c (DEFAULT_TABLE table=test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:22.596498 21278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8837e0d97ac40dea9fe697cf15ab65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.598002 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.584211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33541 (local address 127.0.0.1:39202)
0812 06:51:22.590311 (+ 6100us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.590329 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.590447 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.591829 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.591837 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.591862 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:22.592024 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.592034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.593798 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.593803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.595566 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.595591 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.597319 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.597325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.597505 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.597535 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.597631 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.597755 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6003,"thread_start_us":6411,"threads_started":1}
I20260812 06:51:22.598124 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.584483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39202 (local address 127.17.69.65:33541)
0812 06:51:22.591212 (+ 6729us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.591220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.591245 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:51:22.591341 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.591348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.591543 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.591710 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:22.592170 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.593617 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.595770 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.597145 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.597287 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.597473 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.597681 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.597842 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6577,"thread_start_us":6453,"threads_started":1}
I20260812 06:51:22.598134 21511 tablet_service.cc:1467] Processing CreateTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c (DEFAULT_TABLE table=test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:22.599472 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8837e0d97ac40dea9fe697cf15ab65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.609874 21046 tablet_service.cc:1467] Processing CreateTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c (DEFAULT_TABLE table=test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:22.611265 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8837e0d97ac40dea9fe697cf15ab65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.614666 21654 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Bootstrap starting.
I20260812 06:51:22.616405 21655 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Bootstrap starting.
I20260812 06:51:22.617972 21654 tablet_bootstrap.cc:654] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.618985 21654 log.cc:824] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:22.619796 21655 tablet_bootstrap.cc:654] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.620937 21655 log.cc:824] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:22.623873 21655 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: No bootstrap required, opened a new log
I20260812 06:51:22.624136 21655 ts_tablet_manager.cc:1397] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 06:51:22.625041 21660 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Bootstrap starting.
I20260812 06:51:22.626883 21654 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: No bootstrap required, opened a new log
I20260812 06:51:22.627117 21654 ts_tablet_manager.cc:1397] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.001s
I20260812 06:51:22.631449 21660 tablet_bootstrap.cc:654] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.632627 21660 log.cc:824] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:22.632488 21655 raft_consensus.cc:348] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.632983 21655 raft_consensus.cc:374] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.633103 21655 raft_consensus.cc:729] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.633734 21655 consensus_queue.cc:260] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.634755 21654 raft_consensus.cc:348] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.634990 21619 heartbeater.cc:503] Master 127.17.69.126:33893 was elected leader, sending a full tablet report...
I20260812 06:51:22.635118 21654 raft_consensus.cc:374] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.635215 21654 raft_consensus.cc:729] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.635707 21654 consensus_queue.cc:260] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.636610 21394 heartbeater.cc:503] Master 127.17.69.126:33893 was elected leader, sending a full tablet report...
I20260812 06:51:22.637248 21654 ts_tablet_manager.cc:1428] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260812 06:51:22.639489 21655 ts_tablet_manager.cc:1428] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260812 06:51:22.642625 21660 tablet_bootstrap.cc:492] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: No bootstrap required, opened a new log
I20260812 06:51:22.642853 21660 ts_tablet_manager.cc:1397] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260812 06:51:22.649220 21660 raft_consensus.cc:348] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.649504 21660 raft_consensus.cc:374] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.649590 21660 raft_consensus.cc:729] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eec83786cc48ef864cd40c910c7881, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.650064 21660 consensus_queue.cc:260] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.651257 21660 ts_tablet_manager.cc:1428] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260812 06:51:22.690026 21664 raft_consensus.cc:482] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:22.690300 21664 raft_consensus.cc:504] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.691908 21664 leader_election.cc:288] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341), c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541)
I20260812 06:51:22.698698 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.693304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:55407 (local address 127.17.69.65:33541)
0812 06:51:22.693493 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.693498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.693669 (+ 171us) server_negotiation.cc:407] Connection header received
0812 06:51:22.693854 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.693858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.693928 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.694020 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:22.694629 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.695392 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.697084 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.697841 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.697994 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.698063 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.698321 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.698462 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:51:22.698747 21650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.692351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:43273 (local address 127.17.69.67:40341)
0812 06:51:22.692575 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.692583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.693048 (+ 465us) server_negotiation.cc:407] Connection header received
0812 06:51:22.693303 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.693308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.693385 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.693449 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:22.694225 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.695083 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.696944 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.697740 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.697858 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.697914 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.698379 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.698500 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:51:22.698856 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.692276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40341 (local address 127.17.69.66:43273)
0812 06:51:22.692922 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.692942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.693093 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.693652 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.693656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.693713 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:22.694056 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.694070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.695211 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.695216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.696778 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.696790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.697958 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.697965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.698154 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.698222 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.698307 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.698449 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":43,"spinlock_wait_cycles":30720,"thread_start_us":354,"threads_started":1}
I20260812 06:51:22.699156 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.693008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33541 (local address 127.17.69.66:55407)
0812 06:51:22.693578 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.693592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.693724 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.694051 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.694053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.694072 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:22.694479 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.694493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.695523 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.695526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.696945 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.696959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.698044 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.698048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.698163 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.698206 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.698276 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.698423 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":4,"spinlock_wait_cycles":10496,"thread_start_us":298,"threads_started":1}
I20260812 06:51:22.699828 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8837e0d97ac40dea9fe697cf15ab65c" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de" is_pre_election: true
I20260812 06:51:22.700047 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8837e0d97ac40dea9fe697cf15ab65c" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881" is_pre_election: true
I20260812 06:51:22.700278 21541 raft_consensus.cc:2454] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:22.700512 21092 raft_consensus.cc:2454] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:22.700901 21199 leader_election.cc:302] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, 4833e71e4623433db68d8e7d5e6570de; no voters:
I20260812 06:51:22.701293 21664 raft_consensus.cc:2781] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:22.701416 21664 raft_consensus.cc:482] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:22.701522 21664 raft_consensus.cc:3037] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:22.703727 21664 raft_consensus.cc:504] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.704246 21664 leader_election.cc:288] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Requested vote from peers 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341), c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541)
I20260812 06:51:22.704605 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8837e0d97ac40dea9fe697cf15ab65c" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de"
I20260812 06:51:22.704677 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8837e0d97ac40dea9fe697cf15ab65c" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881"
I20260812 06:51:22.704777 21541 raft_consensus.cc:3037] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:22.704854 21092 raft_consensus.cc:3037] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:22.706956 21541 raft_consensus.cc:2454] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:22.707078 21092 raft_consensus.cc:2454] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:22.707350 21199 leader_election.cc:302] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, 4833e71e4623433db68d8e7d5e6570de; no voters:
I20260812 06:51:22.707625 21664 raft_consensus.cc:2781] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:22.708038 21664 raft_consensus.cc:686] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 1 LEADER]: Becoming Leader. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Running, Role: LEADER
I20260812 06:51:22.708557 21664 consensus_queue.cc:237] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } }
I20260812 06:51:22.712954 20787 catalog_manager.cc:5495] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } health_report { overall_health: UNKNOWN } } }
W20260812 06:51:22.736961 21620 tablet.cc:2369] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:51:22.737799 21395 tablet.cc:2369] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:51:22.739538 21162 tablet.cc:2369] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:51:22 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517480, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:22.777094 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.764597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.17.69.126:33893)
0812 06:51:22.764766 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:22.764771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:22.765341 (+ 570us) server_negotiation.cc:407] Connection header received
0812 06:51:22.765499 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:22.765504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:22.765570 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:22.765649 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:22.766840 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.767574 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.768771 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.769196 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.769303 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.772318 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:22.772323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:22.772326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:22.772354 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:22.774562 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.775204 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.775209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.775211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.775345 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:22.775690 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:22.775694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:22.775696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:22.775952 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:22.776100 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:22.776489 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 06:51:22.776763 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":61}
I20260812 06:51:22.777093 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:22.764661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:33893 (local address 127.0.0.1:38612)
0812 06:51:22.765240 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:22.765255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:22.765389 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:22.765698 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:22.765702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:22.766449 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:22.766700 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.766710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.767703 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.767707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.768639 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:22.768651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:22.769364 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:22.769371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:22.769530 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:22.770073 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:22.770090 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:22.772152 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:22.774683 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.774696 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.774704 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.775098 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.775447 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:22.775451 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:22.775455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:22.775591 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:22.776080 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:22.776088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:22.776245 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 06:51:22.776391 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 06:51:22.776637 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":308,"threads_started":1}
I20260812 06:51:22.781664 20787 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:38612:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:51:22.782608 20787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:22.788642 21046 tablet_service.cc:1467] Processing CreateTablet for tablet 8c22757772ab4bc08bede51121439672 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2979b09dd02441b0a778b67aff019d30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:22.788892 21278 tablet_service.cc:1467] Processing CreateTablet for tablet 8c22757772ab4bc08bede51121439672 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2979b09dd02441b0a778b67aff019d30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:22.789628 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c22757772ab4bc08bede51121439672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.789628 21278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c22757772ab4bc08bede51121439672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.789458 21511 tablet_service.cc:1467] Processing CreateTablet for tablet 8c22757772ab4bc08bede51121439672 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2979b09dd02441b0a778b67aff019d30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:51:22.790107 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c22757772ab4bc08bede51121439672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:22.795495 21655 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: Bootstrap starting.
I20260812 06:51:22.796599 21660 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: Bootstrap starting.
I20260812 06:51:22.797027 21655 tablet_bootstrap.cc:654] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.797204 21654 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: Bootstrap starting.
I20260812 06:51:22.798188 21660 tablet_bootstrap.cc:654] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.798506 21654 tablet_bootstrap.cc:654] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:22.800560 21660 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: No bootstrap required, opened a new log
I20260812 06:51:22.800637 21655 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: No bootstrap required, opened a new log
I20260812 06:51:22.800710 21660 ts_tablet_manager.cc:1397] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260812 06:51:22.800765 21655 ts_tablet_manager.cc:1397] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260812 06:51:22.800839 21654 tablet_bootstrap.cc:492] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: No bootstrap required, opened a new log
I20260812 06:51:22.800977 21654 ts_tablet_manager.cc:1397] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 06:51:22.801477 21655 raft_consensus.cc:348] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.801477 21660 raft_consensus.cc:348] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.801604 21654 raft_consensus.cc:348] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.801723 21655 raft_consensus.cc:374] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.801723 21660 raft_consensus.cc:374] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.801812 21655 raft_consensus.cc:729] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.801824 21654 raft_consensus.cc:374] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:22.801819 21660 raft_consensus.cc:729] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eec83786cc48ef864cd40c910c7881, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.801915 21654 raft_consensus.cc:729] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Initialized, Role: FOLLOWER
I20260812 06:51:22.802027 21655 consensus_queue.cc:260] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.802027 21660 consensus_queue.cc:260] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.802135 21654 consensus_queue.cc:260] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:22.802594 21660 ts_tablet_manager.cc:1428] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260812 06:51:22.802594 21655 ts_tablet_manager.cc:1428] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 06:51:22.802983 21663 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:22.803330 21663 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:22.802598 21664 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:22.803504 21664 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:22.803617 21667 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:22.803912 21654 ts_tablet_manager.cc:1428] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260812 06:51:22.803918 21667 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:23.161170 21664 raft_consensus.cc:482] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:23.161381 21664 raft_consensus.cc:504] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.161983 21664 leader_election.cc:288] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341)
I20260812 06:51:23.162530 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c22757772ab4bc08bede51121439672" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881" is_pre_election: true
I20260812 06:51:23.162616 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c22757772ab4bc08bede51121439672" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de" is_pre_election: true
I20260812 06:51:23.162771 21092 raft_consensus.cc:2454] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:23.162886 21541 raft_consensus.cc:2454] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:23.163195 21199 leader_election.cc:302] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:23.163465 21664 raft_consensus.cc:2781] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:23.163552 21664 raft_consensus.cc:482] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:23.163650 21664 raft_consensus.cc:3037] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.165807 21664 raft_consensus.cc:504] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.166281 21664 leader_election.cc:288] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Requested vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341)
I20260812 06:51:23.166712 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c22757772ab4bc08bede51121439672" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881"
I20260812 06:51:23.166733 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c22757772ab4bc08bede51121439672" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de"
I20260812 06:51:23.166894 21092 raft_consensus.cc:3037] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.166911 21541 raft_consensus.cc:3037] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.168709 21541 raft_consensus.cc:2454] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:23.168812 21092 raft_consensus.cc:2454] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:23.169072 21199 leader_election.cc:302] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, 4833e71e4623433db68d8e7d5e6570de; no voters:
I20260812 06:51:23.169296 21664 raft_consensus.cc:2781] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:23.169452 21664 raft_consensus.cc:686] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 LEADER]: Becoming Leader. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Running, Role: LEADER
I20260812 06:51:23.169690 21664 consensus_queue.cc:237] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.171291 21672 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: TxnStatusTablet state changed. Reason: New leader 28c6b135b04d43bdb3f2e87342575c47. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.171530 21672 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:23.172734 20784 catalog_manager.cc:5495] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } health_report { overall_health: UNKNOWN } } }
I20260812 06:51:23.178715 21698 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:51:23.178958 21698 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:51:23.256253 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.250096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.17.69.66:46493)
0812 06:51:23.250644 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.250649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.250664 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:51:23.250740 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.250744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.250823 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.250952 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.251347 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.252146 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.254039 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.254518 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.254615 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.255164 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.255721 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.255819 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.255951 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.256013 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":258,"threads_started":1}
I20260812 06:51:23.256482 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.249962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46493 (local address 127.0.0.1:34150)
0812 06:51:23.250177 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.250194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.250326 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.250999 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.251005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.251018 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:23.251201 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.251211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.252298 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.252304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.253893 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.253911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.254667 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.254674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.254896 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.254970 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.255825 (+ 855us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.255846 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.255936 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.256093 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:51:23.262395 21664 consensus_queue.cc:1035] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.263813 21541 raft_consensus.cc:1261] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Refusing update from remote peer 28c6b135b04d43bdb3f2e87342575c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:23.266196 21672 consensus_queue.cc:1035] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.266796 21092 raft_consensus.cc:1261] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Refusing update from remote peer 28c6b135b04d43bdb3f2e87342575c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:23.268451 21664 consensus_queue.cc:1035] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.274062 21708 consensus_queue.cc:1035] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.292929 21667 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: TxnStatusTablet state changed. Reason: New leader 28c6b135b04d43bdb3f2e87342575c47. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.293277 21667 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:23.295284 21707 mvcc.cc:205] Tried to move back new op lower bound from 7317575611438952448 to 7317575611065675776. Current Snapshot: MvccSnapshot[applied={T|T < 7317575611438952448}]
I20260812 06:51:23.297145 21663 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: TxnStatusTablet state changed. Reason: New leader 28c6b135b04d43bdb3f2e87342575c47. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.297438 21663 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:23.300624 21712 mvcc.cc:205] Tried to move back new op lower bound from 7317575611438952448 to 7317575611065675776. Current Snapshot: MvccSnapshot[applied={T|T < 7317575611438952448}]
I20260812 06:51:23.301241 21713 mvcc.cc:205] Tried to move back new op lower bound from 7317575611438952448 to 7317575611065675776. Current Snapshot: MvccSnapshot[applied={T|T < 7317575611438952448}]
I20260812 06:51:23.301285 21708 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.301597 21708 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:23.304092 21667 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.304303 21667 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P c3eec83786cc48ef864cd40c910c7881: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:23.305462 21664 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.305685 21664 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 28c6b135b04d43bdb3f2e87342575c47: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:51:23.308357 21663 tablet_replica.cc:439] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } } }
I20260812 06:51:23.308578 21663 tablet_replica.cc:442] T 8c22757772ab4bc08bede51121439672 P 4833e71e4623433db68d8e7d5e6570de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:51:23.319351 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.313561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.17.69.66:46493)
0812 06:51:23.313703 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.313707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.314862 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 06:51:23.315044 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.315048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.315108 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.315184 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.315617 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.316420 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.317771 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.318204 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.318312 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.318711 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.318805 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.318884 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.319021 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.319097 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:51:23.335106 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.329026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:41867 (local address 127.17.69.66:46493)
0812 06:51:23.329202 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.329207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.329613 (+ 406us) server_negotiation.cc:407] Connection header received
0812 06:51:23.329792 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.329796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.329878 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.329943 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.330496 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.331199 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.333243 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.333640 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.333865 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.334410 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.334509 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.334579 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.334732 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.334789 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:51:23.335289 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.328922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46493 (local address 127.17.69.66:41867)
0812 06:51:23.329519 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.329537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.329682 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.329983 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.329988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.330001 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:23.330304 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.330315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.331330 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.331336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.333111 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.333127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.333910 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.333940 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.334120 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.334194 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.334606 (+ 412us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.334624 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.334711 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.334843 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":267,"threads_started":1}
I20260812 06:51:23.393834 20784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:23.394133 20784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:23.397481 20784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Sending DeleteTablet for 3 replicas of tablet d8837e0d97ac40dea9fe697cf15ab65c
I20260812 06:51:23.398702 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:23 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 06:51:23.398795 21511 tablet_service.cc:1514] Processing DeleteTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:23 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:55788
I20260812 06:51:23.399680 20784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:51:23.400293 21278 tablet_service.cc:1514] Processing DeleteTablet for tablet d8837e0d97ac40dea9fe697cf15ab65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:23 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:34142
I20260812 06:51:23.400422 21737 tablet_replica.cc:330] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: stopping tablet replica
W20260812 06:51:23.400398 20784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:23.400580 21738 tablet_replica.cc:330] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: stopping tablet replica
I20260812 06:51:23.400858 21737 raft_consensus.cc:2229] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:23.400928 21738 raft_consensus.cc:2229] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:23.401288 21738 raft_consensus.cc:2258] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:23.401325 21737 raft_consensus.cc:2258] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:23.403128 21738 ts_tablet_manager.cc:1905] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:23.403129 21737 ts_tablet_manager.cc:1905] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:23.407784 21046 tablet_service.cc:1467] Processing CreateTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 (DEFAULT_TABLE table=test-table [id=947f9e22eb2741e8b34f494e8ba839fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:23.408234 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605657b92c6e436fb2d8b26f470d3aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:23.408321 21276 tablet_service.cc:1467] Processing CreateTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 (DEFAULT_TABLE table=test-table [id=947f9e22eb2741e8b34f494e8ba839fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:23.408854 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605657b92c6e436fb2d8b26f470d3aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:23.410374 21737 ts_tablet_manager.cc:1918] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:23.410530 21737 log.cc:1196] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/d8837e0d97ac40dea9fe697cf15ab65c
I20260812 06:51:23.411015 21737 ts_tablet_manager.cc:1939] T d8837e0d97ac40dea9fe697cf15ab65c P c3eec83786cc48ef864cd40c910c7881: Deleting consensus metadata
I20260812 06:51:23.413316 20760 catalog_manager.cc:4841] TS c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541): tablet d8837e0d97ac40dea9fe697cf15ab65c (table test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]) successfully deleted
I20260812 06:51:23.413537 21511 tablet_service.cc:1467] Processing CreateTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 (DEFAULT_TABLE table=test-table [id=947f9e22eb2741e8b34f494e8ba839fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:23.413942 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605657b92c6e436fb2d8b26f470d3aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:23.415459 21739 tablet_replica.cc:330] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: stopping tablet replica
I20260812 06:51:23.415869 21739 raft_consensus.cc:2229] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:23.416527 21739 raft_consensus.cc:2258] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:23.418767 21739 ts_tablet_manager.cc:1905] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:23.422116 21741 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Bootstrap starting.
I20260812 06:51:23.422096 21738 ts_tablet_manager.cc:1918] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:23.422247 21738 log.cc:1196] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/d8837e0d97ac40dea9fe697cf15ab65c
I20260812 06:51:23.423930 21738 ts_tablet_manager.cc:1939] T d8837e0d97ac40dea9fe697cf15ab65c P 4833e71e4623433db68d8e7d5e6570de: Deleting consensus metadata
I20260812 06:51:23.424858 21741 tablet_bootstrap.cc:654] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:23.426183 20760 catalog_manager.cc:4841] TS 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341): tablet d8837e0d97ac40dea9fe697cf15ab65c (table test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]) successfully deleted
I20260812 06:51:23.426245 21740 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Bootstrap starting.
I20260812 06:51:23.426779 21743 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Bootstrap starting.
I20260812 06:51:23.428416 21743 tablet_bootstrap.cc:654] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:23.429185 21740 tablet_bootstrap.cc:654] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:23.429862 21739 ts_tablet_manager.cc:1918] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:23.430048 21739 log.cc:1196] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/d8837e0d97ac40dea9fe697cf15ab65c
I20260812 06:51:23.430785 21739 ts_tablet_manager.cc:1939] T d8837e0d97ac40dea9fe697cf15ab65c P 28c6b135b04d43bdb3f2e87342575c47: Deleting consensus metadata
I20260812 06:51:23.432652 20761 catalog_manager.cc:4841] TS 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493): tablet d8837e0d97ac40dea9fe697cf15ab65c (table test-table [id=fe12c91c3a544fb884b2ee6f8e5e15b7]) successfully deleted
I20260812 06:51:23.434154 21741 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: No bootstrap required, opened a new log
I20260812 06:51:23.434391 21741 ts_tablet_manager.cc:1397] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260812 06:51:23.435343 21741 raft_consensus.cc:348] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.435794 21741 raft_consensus.cc:374] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:23.435942 21741 raft_consensus.cc:729] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Initialized, Role: FOLLOWER
I20260812 06:51:23.436192 21743 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: No bootstrap required, opened a new log
I20260812 06:51:23.436336 21743 ts_tablet_manager.cc:1397] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260812 06:51:23.436318 21741 consensus_queue.cc:260] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.437052 21743 raft_consensus.cc:348] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.437283 21743 raft_consensus.cc:374] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:23.437368 21743 raft_consensus.cc:729] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Initialized, Role: FOLLOWER
I20260812 06:51:23.438093 21743 consensus_queue.cc:260] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.438722 21743 ts_tablet_manager.cc:1428] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:23.440022 21740 tablet_bootstrap.cc:492] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: No bootstrap required, opened a new log
I20260812 06:51:23.440155 21740 ts_tablet_manager.cc:1397] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260812 06:51:23.439512 21741 ts_tablet_manager.cc:1428] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260812 06:51:23.440860 21740 raft_consensus.cc:348] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.441148 21740 raft_consensus.cc:374] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:23.441283 21740 raft_consensus.cc:729] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eec83786cc48ef864cd40c910c7881, State: Initialized, Role: FOLLOWER
I20260812 06:51:23.441597 21740 consensus_queue.cc:260] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.442394 21740 ts_tablet_manager.cc:1428] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 06:51:23.820791 21663 raft_consensus.cc:482] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:23.821007 21663 raft_consensus.cc:504] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.822628 21663 leader_election.cc:288] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493)
I20260812 06:51:23.840292 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.823177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:46931 (local address 127.17.69.65:33541)
0812 06:51:23.834022 (+ 10845us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.834027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.834045 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:51:23.834167 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.834171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.834228 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.834676 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:23.835220 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.836399 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.838243 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.839248 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.839439 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.839509 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.839823 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.839954 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10711,"thread_start_us":10765,"threads_started":1}
I20260812 06:51:23.840332 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.822935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33541 (local address 127.17.69.67:46931)
0812 06:51:23.833027 (+ 10092us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.833047 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.833196 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.834417 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.834422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.834440 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:23.834990 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.835008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.836515 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.836521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.838104 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.838116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.839456 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.839463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.839644 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.839682 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.839790 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.839920 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9960,"mutex_wait_us":146,"thread_start_us":10579,"threads_started":1}
I20260812 06:51:23.841248 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605657b92c6e436fb2d8b26f470d3aa4" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881" is_pre_election: true
I20260812 06:51:23.841375 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.823396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:59785 (local address 127.17.69.66:46493)
0812 06:51:23.823615 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.823622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.833916 (+ 10294us) server_negotiation.cc:407] Connection header received
0812 06:51:23.834136 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.834142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.834223 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.834377 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:23.835067 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.836293 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.838873 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.840047 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.840207 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.840283 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.840944 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.841089 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:51:23.841394 21748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.822961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46493 (local address 127.17.69.67:59785)
0812 06:51:23.833798 (+ 10837us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.833815 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.833943 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.834388 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.834393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.834421 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:23.834876 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.834890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.836461 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.836466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.838689 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.838706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.840588 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.840595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.840756 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.840790 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.840871 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.840986 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10720,"mutex_wait_us":75,"thread_start_us":9801,"threads_started":1}
I20260812 06:51:23.841542 21092 raft_consensus.cc:2454] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 0.
I20260812 06:51:23.842106 21427 leader_election.cc:302] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4833e71e4623433db68d8e7d5e6570de, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:23.842713 21663 raft_consensus.cc:2781] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:23.842845 21663 raft_consensus.cc:482] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:23.842726 21304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605657b92c6e436fb2d8b26f470d3aa4" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c6b135b04d43bdb3f2e87342575c47" is_pre_election: true
I20260812 06:51:23.842929 21663 raft_consensus.cc:3037] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.843115 21304 raft_consensus.cc:2454] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 0.
I20260812 06:51:23.845182 21663 raft_consensus.cc:504] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.845705 21663 leader_election.cc:288] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 election: Requested vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493)
I20260812 06:51:23.846266 21304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605657b92c6e436fb2d8b26f470d3aa4" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c6b135b04d43bdb3f2e87342575c47"
I20260812 06:51:23.846284 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605657b92c6e436fb2d8b26f470d3aa4" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881"
I20260812 06:51:23.846484 21304 raft_consensus.cc:3037] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.846524 21092 raft_consensus.cc:3037] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:23.848682 21304 raft_consensus.cc:2454] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 1.
I20260812 06:51:23.849503 21092 raft_consensus.cc:2454] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 1.
I20260812 06:51:23.849907 21427 leader_election.cc:302] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, 4833e71e4623433db68d8e7d5e6570de, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:23.850193 21663 raft_consensus.cc:2781] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:23.850660 21663 raft_consensus.cc:686] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 1 LEADER]: Becoming Leader. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Running, Role: LEADER
I20260812 06:51:23.851056 21663 consensus_queue.cc:237] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:23.854583 20785 catalog_manager.cc:5495] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de reported cstate change: term changed from 0 to 1, leader changed from <none> to 4833e71e4623433db68d8e7d5e6570de (127.17.69.67). New cstate: current_term: 1 leader_uuid: "4833e71e4623433db68d8e7d5e6570de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } health_report { overall_health: HEALTHY } } }
I20260812 06:51:23.910425 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.903925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.17.69.67:40341)
0812 06:51:23.904424 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.904430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.904939 (+ 509us) server_negotiation.cc:407] Connection header received
0812 06:51:23.905168 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.905174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.905276 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.905444 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.905940 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.906768 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.908066 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.908488 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.908622 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.909050 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.909669 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.909805 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.909921 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.910004 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":271,"threads_started":1}
I20260812 06:51:23.925866 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.920517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:38565 (local address 127.17.69.66:46493)
0812 06:51:23.920662 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.920666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.921189 (+ 523us) server_negotiation.cc:407] Connection header received
0812 06:51:23.921384 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.921388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.921456 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.921540 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.922036 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.922802 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.924120 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.924596 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.924711 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.925136 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.925238 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.925325 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.925461 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.925553 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:51:23.925925 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.920443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46493 (local address 127.17.69.67:38565)
0812 06:51:23.921079 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.921095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.921246 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.921558 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.921562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.921574 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:23.921878 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.921887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.922932 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.922938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.923979 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.923991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.924778 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.924785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.924906 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.924965 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.925340 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.925352 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.925417 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.925547 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":344,"threads_started":1}
I20260812 06:51:23.934749 21089 raft_consensus.cc:1261] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Refusing update from remote peer 4833e71e4623433db68d8e7d5e6570de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:23.934870 21304 raft_consensus.cc:1261] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Refusing update from remote peer 4833e71e4623433db68d8e7d5e6570de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:23.935629 21663 consensus_queue.cc:1035] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.935921 21751 consensus_queue.cc:1035] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:23.972591 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.963793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:39869 (local address 127.17.69.67:40341)
0812 06:51:23.963944 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:23.963950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:23.966817 (+ 2867us) server_negotiation.cc:407] Connection header received
0812 06:51:23.966936 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:23.966941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:23.967018 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:23.967106 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:23.967603 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.968420 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.971001 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.971444 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.971544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.971995 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.972089 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.972148 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:23.972300 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 06:51:23.972362 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:51:23.972810 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:23.963343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40341 (local address 127.17.69.66:39869)
0812 06:51:23.966720 (+ 3377us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:23.966736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:23.966839 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:23.967110 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:23.967113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:23.967122 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:51:23.967454 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.967467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.968569 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.968575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.970830 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:23.970848 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:23.971609 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:23.971616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:23.971808 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:23.971865 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:51:23.972182 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:51:23.972194 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 06:51:23.972288 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:51:23.972423 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3257,"thread_start_us":3148,"threads_started":1}
W20260812 06:51:23.973932 21712 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:51:25.581255 20785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:25.581491 20785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:25.585371 20785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Sending DeleteTablet for 3 replicas of tablet 605657b92c6e436fb2d8b26f470d3aa4
I20260812 06:51:25.586045 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 06:51:25.586091 21276 tablet_service.cc:1514] Processing DeleteTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:34142
I20260812 06:51:25.586153 21511 tablet_service.cc:1514] Processing DeleteTablet for tablet 605657b92c6e436fb2d8b26f470d3aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:55788
I20260812 06:51:25.598392 21824 tablet_replica.cc:330] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: stopping tablet replica
I20260812 06:51:25.598405 21827 tablet_replica.cc:330] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: stopping tablet replica
I20260812 06:51:25.598573 21824 raft_consensus.cc:2229] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:25.598610 21827 raft_consensus.cc:2229] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:25.598794 21824 raft_consensus.cc:2258] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.598829 21827 raft_consensus.cc:2258] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.599839 21827 ts_tablet_manager.cc:1905] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:25.599844 21824 ts_tablet_manager.cc:1905] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:25.601522 21830 tablet_replica.cc:330] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: stopping tablet replica
I20260812 06:51:25.601713 21830 raft_consensus.cc:2229] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:25.602231 21830 raft_consensus.cc:2258] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.604976 21824 ts_tablet_manager.cc:1918] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:25.605086 21824 log.cc:1196] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/605657b92c6e436fb2d8b26f470d3aa4
I20260812 06:51:25.605532 21824 ts_tablet_manager.cc:1939] T 605657b92c6e436fb2d8b26f470d3aa4 P c3eec83786cc48ef864cd40c910c7881: Deleting consensus metadata
I20260812 06:51:25.605849 21830 ts_tablet_manager.cc:1905] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:25.605953 21827 ts_tablet_manager.cc:1918] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:25.606076 21827 log.cc:1196] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/605657b92c6e436fb2d8b26f470d3aa4
I20260812 06:51:25.606544 21827 ts_tablet_manager.cc:1939] T 605657b92c6e436fb2d8b26f470d3aa4 P 28c6b135b04d43bdb3f2e87342575c47: Deleting consensus metadata
I20260812 06:51:25.606622 20760 catalog_manager.cc:4841] TS c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541): tablet 605657b92c6e436fb2d8b26f470d3aa4 (table test-table [id=947f9e22eb2741e8b34f494e8ba839fd]) successfully deleted
I20260812 06:51:25.607546 20761 catalog_manager.cc:4841] TS 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493): tablet 605657b92c6e436fb2d8b26f470d3aa4 (table test-table [id=947f9e22eb2741e8b34f494e8ba839fd]) successfully deleted
I20260812 06:51:25.611264 21830 ts_tablet_manager.cc:1918] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:25.611382 21830 log.cc:1196] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/605657b92c6e436fb2d8b26f470d3aa4
I20260812 06:51:25.611809 21830 ts_tablet_manager.cc:1939] T 605657b92c6e436fb2d8b26f470d3aa4 P 4833e71e4623433db68d8e7d5e6570de: Deleting consensus metadata
I20260812 06:51:25.612771 20760 catalog_manager.cc:4841] TS 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341): tablet 605657b92c6e436fb2d8b26f470d3aa4 (table test-table [id=947f9e22eb2741e8b34f494e8ba839fd]) successfully deleted
Aug 12 06:51:25 dist-test-slave-6zbq krb5kdc[20694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:51:25 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:25.647336 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:25.633815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.17.69.126:33893)
0812 06:51:25.634401 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:25.634407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:25.634428 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:51:25.634561 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:25.634566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:25.634632 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:25.634722 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:25.635990 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:25.636790 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:25.637952 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:25.638429 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:25.638523 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:25.642150 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:25.642155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:25.642159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:25.642190 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:25.644737 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:25.645403 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:25.645409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:25.645412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:25.645537 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:25.645891 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:25.645896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:25.645899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:25.646194 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:25.646446 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:25.646774 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 06:51:25.646990 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":284,"threads_started":1}
I20260812 06:51:25.650651 20785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:25.651232 20785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:25.657506 21046 tablet_service.cc:1467] Processing CreateTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 (DEFAULT_TABLE table=test-table [id=41a6b06b512c455facf629b8a74ff39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.657649 21276 tablet_service.cc:1467] Processing CreateTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 (DEFAULT_TABLE table=test-table [id=41a6b06b512c455facf629b8a74ff39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.657823 21511 tablet_service.cc:1467] Processing CreateTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 (DEFAULT_TABLE table=test-table [id=41a6b06b512c455facf629b8a74ff39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.657959 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c74fee9d514ac7bf3197e3811f2a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.658063 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c74fee9d514ac7bf3197e3811f2a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.658282 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c74fee9d514ac7bf3197e3811f2a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.664047 21840 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Bootstrap starting.
I20260812 06:51:25.665426 21840 tablet_bootstrap.cc:654] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.667701 21845 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Bootstrap starting.
I20260812 06:51:25.668246 21841 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Bootstrap starting.
I20260812 06:51:25.668299 21840 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: No bootstrap required, opened a new log
I20260812 06:51:25.668422 21840 ts_tablet_manager.cc:1397] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260812 06:51:25.669188 21840 raft_consensus.cc:348] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.669442 21840 raft_consensus.cc:374] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.669543 21840 raft_consensus.cc:729] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.669915 21841 tablet_bootstrap.cc:654] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.669829 21840 consensus_queue.cc:260] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.670475 21840 ts_tablet_manager.cc:1428] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 06:51:25.672454 21845 tablet_bootstrap.cc:654] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.674304 21841 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: No bootstrap required, opened a new log
I20260812 06:51:25.674469 21841 ts_tablet_manager.cc:1397] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260812 06:51:25.674564 21845 tablet_bootstrap.cc:492] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: No bootstrap required, opened a new log
I20260812 06:51:25.674711 21845 ts_tablet_manager.cc:1397] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260812 06:51:25.675117 21841 raft_consensus.cc:348] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.675333 21841 raft_consensus.cc:374] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.675441 21841 raft_consensus.cc:729] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eec83786cc48ef864cd40c910c7881, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.675386 21845 raft_consensus.cc:348] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.675637 21845 raft_consensus.cc:374] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.675735 21845 raft_consensus.cc:729] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.675695 21841 consensus_queue.cc:260] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.675966 21845 consensus_queue.cc:260] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.676625 21841 ts_tablet_manager.cc:1428] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 06:51:25.676952 21845 ts_tablet_manager.cc:1428] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:25.781379 21802 raft_consensus.cc:482] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:25.781585 21802 raft_consensus.cc:504] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.782275 21802 leader_election.cc:288] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341)
I20260812 06:51:25.782809 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c74fee9d514ac7bf3197e3811f2a77" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881" is_pre_election: true
I20260812 06:51:25.782869 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c74fee9d514ac7bf3197e3811f2a77" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de" is_pre_election: true
I20260812 06:51:25.783070 21092 raft_consensus.cc:2454] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:25.783145 21541 raft_consensus.cc:2454] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 0.
I20260812 06:51:25.783521 21199 leader_election.cc:302] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:25.783780 21802 raft_consensus.cc:2781] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:25.783910 21802 raft_consensus.cc:482] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:25.783991 21802 raft_consensus.cc:3037] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.786134 21802 raft_consensus.cc:504] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.786624 21802 leader_election.cc:288] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Requested vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341)
I20260812 06:51:25.787133 21541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c74fee9d514ac7bf3197e3811f2a77" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4833e71e4623433db68d8e7d5e6570de"
I20260812 06:51:25.787138 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c74fee9d514ac7bf3197e3811f2a77" candidate_uuid: "28c6b135b04d43bdb3f2e87342575c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881"
I20260812 06:51:25.787328 21092 raft_consensus.cc:3037] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.787345 21541 raft_consensus.cc:3037] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.790126 21541 raft_consensus.cc:2454] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:25.790149 21092 raft_consensus.cc:2454] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28c6b135b04d43bdb3f2e87342575c47 in term 1.
I20260812 06:51:25.790563 21199 leader_election.cc:302] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c6b135b04d43bdb3f2e87342575c47, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:25.790829 21802 raft_consensus.cc:2781] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:25.791005 21802 raft_consensus.cc:686] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 LEADER]: Becoming Leader. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Running, Role: LEADER
I20260812 06:51:25.791299 21802 consensus_queue.cc:237] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.794523 20785 catalog_manager.cc:5495] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "28c6b135b04d43bdb3f2e87342575c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } health_report { overall_health: UNKNOWN } } }
I20260812 06:51:25.825460 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:25.820809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.17.69.66:46493)
0812 06:51:25.821406 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:25.821428 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:25.821448 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:51:25.821504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:25.821509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:25.821585 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:25.821673 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:25.821997 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:25.822795 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:25.824059 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:25.824458 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:25.824548 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:25.824899 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:25.824999 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:25.825062 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:25.825156 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 06:51:25.825212 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":390,"threads_started":1}
I20260812 06:51:25.833841 21092 raft_consensus.cc:1261] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Refusing update from remote peer 28c6b135b04d43bdb3f2e87342575c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:25.833904 21541 raft_consensus.cc:1261] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Refusing update from remote peer 28c6b135b04d43bdb3f2e87342575c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:25.834421 21802 consensus_queue.cc:1035] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:25.834679 21785 consensus_queue.cc:1035] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:25.873603 20785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:25.873811 20785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:25.876284 20785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Sending DeleteTablet for 3 replicas of tablet 05c74fee9d514ac7bf3197e3811f2a77
I20260812 06:51:25.876973 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 06:51:25.877067 21511 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:55788
I20260812 06:51:25.877171 21276 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c74fee9d514ac7bf3197e3811f2a77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:25 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:34142
I20260812 06:51:25.877249 21824 tablet_replica.cc:330] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: stopping tablet replica
I20260812 06:51:25.877436 21830 tablet_replica.cc:330] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: stopping tablet replica
I20260812 06:51:25.877439 21824 raft_consensus.cc:2229] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:25.877472 21827 tablet_replica.cc:330] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: stopping tablet replica
I20260812 06:51:25.877609 21824 raft_consensus.cc:2258] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.877609 21830 raft_consensus.cc:2229] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:25.877650 21827 raft_consensus.cc:2229] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:25.877801 21830 raft_consensus.cc:2258] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.877897 21827 raft_consensus.cc:2258] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:25.878620 21824 ts_tablet_manager.cc:1905] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:25.878512 20785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 06:51:25.879079 21827 ts_tablet_manager.cc:1905] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 06:51:25.879150 20785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:25.879361 21830 ts_tablet_manager.cc:1905] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:25.884958 21824 ts_tablet_manager.cc:1918] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:25.885090 21824 log.cc:1196] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/05c74fee9d514ac7bf3197e3811f2a77
I20260812 06:51:25.885398 21830 ts_tablet_manager.cc:1918] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:25.885488 21824 ts_tablet_manager.cc:1939] T 05c74fee9d514ac7bf3197e3811f2a77 P c3eec83786cc48ef864cd40c910c7881: Deleting consensus metadata
I20260812 06:51:25.885543 21830 log.cc:1196] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/05c74fee9d514ac7bf3197e3811f2a77
I20260812 06:51:25.885931 21830 ts_tablet_manager.cc:1939] T 05c74fee9d514ac7bf3197e3811f2a77 P 4833e71e4623433db68d8e7d5e6570de: Deleting consensus metadata
I20260812 06:51:25.886050 21827 ts_tablet_manager.cc:1918] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:51:25.886237 21827 log.cc:1196] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/05c74fee9d514ac7bf3197e3811f2a77
I20260812 06:51:25.886543 20760 catalog_manager.cc:4841] TS c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541): tablet 05c74fee9d514ac7bf3197e3811f2a77 (table test-table [id=41a6b06b512c455facf629b8a74ff39d]) successfully deleted
I20260812 06:51:25.886893 21827 ts_tablet_manager.cc:1939] T 05c74fee9d514ac7bf3197e3811f2a77 P 28c6b135b04d43bdb3f2e87342575c47: Deleting consensus metadata
I20260812 06:51:25.886935 20760 catalog_manager.cc:4841] TS 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341): tablet 05c74fee9d514ac7bf3197e3811f2a77 (table test-table [id=41a6b06b512c455facf629b8a74ff39d]) successfully deleted
I20260812 06:51:25.887887 20761 catalog_manager.cc:4841] TS 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493): tablet 05c74fee9d514ac7bf3197e3811f2a77 (table test-table [id=41a6b06b512c455facf629b8a74ff39d]) successfully deleted
I20260812 06:51:25.892704 21046 tablet_service.cc:1467] Processing CreateTablet for tablet ca1440e803e946c28739eb37c13e92ab (DEFAULT_TABLE table=test-table [id=6587ef3cad4d43b799f3c231bd0f335a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.893076 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1440e803e946c28739eb37c13e92ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.893142 21511 tablet_service.cc:1467] Processing CreateTablet for tablet ca1440e803e946c28739eb37c13e92ab (DEFAULT_TABLE table=test-table [id=6587ef3cad4d43b799f3c231bd0f335a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.893498 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1440e803e946c28739eb37c13e92ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.897699 21840 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Bootstrap starting.
I20260812 06:51:25.898058 21276 tablet_service.cc:1467] Processing CreateTablet for tablet ca1440e803e946c28739eb37c13e92ab (DEFAULT_TABLE table=test-table [id=6587ef3cad4d43b799f3c231bd0f335a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:25.898471 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1440e803e946c28739eb37c13e92ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:25.898953 21841 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Bootstrap starting.
I20260812 06:51:25.899252 21840 tablet_bootstrap.cc:654] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.900418 21841 tablet_bootstrap.cc:654] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.904762 21845 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Bootstrap starting.
I20260812 06:51:25.906126 21840 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: No bootstrap required, opened a new log
I20260812 06:51:25.906213 21845 tablet_bootstrap.cc:654] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:25.906283 21840 ts_tablet_manager.cc:1397] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20260812 06:51:25.906877 21840 raft_consensus.cc:348] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.907095 21840 raft_consensus.cc:374] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.907191 21840 raft_consensus.cc:729] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.907457 21840 consensus_queue.cc:260] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.908300 21845 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: No bootstrap required, opened a new log
I20260812 06:51:25.908509 21845 ts_tablet_manager.cc:1397] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 06:51:25.909300 21845 raft_consensus.cc:348] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.909241 21840 ts_tablet_manager.cc:1428] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260812 06:51:25.909683 21845 raft_consensus.cc:374] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.909832 21845 raft_consensus.cc:729] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c6b135b04d43bdb3f2e87342575c47, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.910063 21845 consensus_queue.cc:260] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.910462 21841 tablet_bootstrap.cc:492] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: No bootstrap required, opened a new log
I20260812 06:51:25.910593 21841 ts_tablet_manager.cc:1397] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260812 06:51:25.911038 21845 ts_tablet_manager.cc:1428] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:25.911266 21841 raft_consensus.cc:348] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.911509 21841 raft_consensus.cc:374] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:25.911607 21841 raft_consensus.cc:729] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3eec83786cc48ef864cd40c910c7881, State: Initialized, Role: FOLLOWER
I20260812 06:51:25.911836 21841 consensus_queue.cc:260] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.912460 21841 ts_tablet_manager.cc:1428] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 06:51:25.980052 21812 raft_consensus.cc:482] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:25.980216 21812 raft_consensus.cc:504] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.980798 21812 leader_election.cc:288] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493)
I20260812 06:51:25.981264 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1440e803e946c28739eb37c13e92ab" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881" is_pre_election: true
I20260812 06:51:25.981261 21308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1440e803e946c28739eb37c13e92ab" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c6b135b04d43bdb3f2e87342575c47" is_pre_election: true
I20260812 06:51:25.981460 21308 raft_consensus.cc:2454] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 0.
I20260812 06:51:25.981472 21092 raft_consensus.cc:2454] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 0.
I20260812 06:51:25.981869 21427 leader_election.cc:302] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4833e71e4623433db68d8e7d5e6570de, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:25.982112 21812 raft_consensus.cc:2781] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:25.982265 21812 raft_consensus.cc:482] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:25.982364 21812 raft_consensus.cc:3037] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.984287 21812 raft_consensus.cc:504] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.984808 21812 leader_election.cc:288] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 election: Requested vote from peers c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541), 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493)
I20260812 06:51:25.985284 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1440e803e946c28739eb37c13e92ab" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3eec83786cc48ef864cd40c910c7881"
I20260812 06:51:25.985289 21308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1440e803e946c28739eb37c13e92ab" candidate_uuid: "4833e71e4623433db68d8e7d5e6570de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c6b135b04d43bdb3f2e87342575c47"
I20260812 06:51:25.985487 21092 raft_consensus.cc:3037] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.985504 21308 raft_consensus.cc:3037] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:25.987598 21092 raft_consensus.cc:2454] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 1.
I20260812 06:51:25.987969 21427 leader_election.cc:302] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4833e71e4623433db68d8e7d5e6570de, c3eec83786cc48ef864cd40c910c7881; no voters:
I20260812 06:51:25.988227 21812 raft_consensus.cc:2781] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:25.988359 21812 raft_consensus.cc:686] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 1 LEADER]: Becoming Leader. State: Replica: 4833e71e4623433db68d8e7d5e6570de, State: Running, Role: LEADER
I20260812 06:51:25.988639 21308 raft_consensus.cc:2454] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4833e71e4623433db68d8e7d5e6570de in term 1.
I20260812 06:51:25.988621 21812 consensus_queue.cc:237] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } }
I20260812 06:51:25.991257 20784 catalog_manager.cc:5495] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de reported cstate change: term changed from 0 to 1, leader changed from <none> to 4833e71e4623433db68d8e7d5e6570de (127.17.69.67). New cstate: current_term: 1 leader_uuid: "4833e71e4623433db68d8e7d5e6570de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4833e71e4623433db68d8e7d5e6570de" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40341 } health_report { overall_health: HEALTHY } } }
I20260812 06:51:26.033154 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:26.028309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.17.69.67:40341)
0812 06:51:26.028887 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:26.028893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:26.028916 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:51:26.028977 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:26.028982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:26.029058 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:26.029158 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:26.029577 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:26.030390 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:26.031687 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:26.032123 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:26.032219 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:26.032576 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:26.032690 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:26.032755 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:26.032856 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 06:51:26.032914 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":318,"threads_started":1}
I20260812 06:51:26.040884 21308 raft_consensus.cc:1261] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Refusing update from remote peer 4833e71e4623433db68d8e7d5e6570de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:26.041002 21092 raft_consensus.cc:1261] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Refusing update from remote peer 4833e71e4623433db68d8e7d5e6570de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:26.041428 21812 consensus_queue.cc:1035] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c6b135b04d43bdb3f2e87342575c47" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:26.041697 21781 consensus_queue.cc:1035] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3eec83786cc48ef864cd40c910c7881" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:51:26.060174 21797 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 06:51:27.632222 20784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:27.632465 20784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:27.636134 20784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32135cbdf20347abb1ae4554a42f2fb5: Sending DeleteTablet for 3 replicas of tablet ca1440e803e946c28739eb37c13e92ab
I20260812 06:51:27.636793 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1440e803e946c28739eb37c13e92ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:27 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 06:51:27.636828 21511 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1440e803e946c28739eb37c13e92ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:27 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:55788
I20260812 06:51:27.636849 21276 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1440e803e946c28739eb37c13e92ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:27 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:34142
I20260812 06:51:27.637455 21935 tablet_replica.cc:330] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: stopping tablet replica
I20260812 06:51:27.638048 21934 tablet_replica.cc:330] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: stopping tablet replica
I20260812 06:51:27.638314 21934 raft_consensus.cc:2229] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:27.638651 21935 raft_consensus.cc:2229] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:27.638795 21934 raft_consensus.cc:2258] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:27.638919 21935 raft_consensus.cc:2258] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:27.639753 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20928
I20260812 06:51:27.639786 21934 ts_tablet_manager.cc:1905] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:27.639899 21935 ts_tablet_manager.cc:1905] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:27.646509 21935 ts_tablet_manager.cc:1918] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:27.646633 21934 ts_tablet_manager.cc:1918] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:27.646679 21935 log.cc:1196] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/ca1440e803e946c28739eb37c13e92ab
I20260812 06:51:27.646801 21934 log.cc:1196] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/ca1440e803e946c28739eb37c13e92ab
I20260812 06:51:27.647202 21935 ts_tablet_manager.cc:1939] T ca1440e803e946c28739eb37c13e92ab P c3eec83786cc48ef864cd40c910c7881: Deleting consensus metadata
I20260812 06:51:27.647316 21934 ts_tablet_manager.cc:1939] T ca1440e803e946c28739eb37c13e92ab P 4833e71e4623433db68d8e7d5e6570de: Deleting consensus metadata
I20260812 06:51:27.648224 20760 catalog_manager.cc:4841] TS c3eec83786cc48ef864cd40c910c7881 (127.17.69.65:33541): tablet ca1440e803e946c28739eb37c13e92ab (table test-table [id=6587ef3cad4d43b799f3c231bd0f335a]) successfully deleted
I20260812 06:51:27.648581 20760 catalog_manager.cc:4841] TS 4833e71e4623433db68d8e7d5e6570de (127.17.69.67:40341): tablet ca1440e803e946c28739eb37c13e92ab (table test-table [id=6587ef3cad4d43b799f3c231bd0f335a]) successfully deleted
I20260812 06:51:27.650795 21936 tablet_replica.cc:330] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: stopping tablet replica
I20260812 06:51:27.650982 21936 raft_consensus.cc:2229] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:27.651196 21936 raft_consensus.cc:2258] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:27.652083 21936 ts_tablet_manager.cc:1905] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:27.656634 21936 ts_tablet_manager.cc:1918] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 06:51:27.656756 21936 log.cc:1196] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/ca1440e803e946c28739eb37c13e92ab
I20260812 06:51:27.657090 21936 ts_tablet_manager.cc:1939] T ca1440e803e946c28739eb37c13e92ab P 28c6b135b04d43bdb3f2e87342575c47: Deleting consensus metadata
I20260812 06:51:27.657925 20761 catalog_manager.cc:4841] TS 28c6b135b04d43bdb3f2e87342575c47 (127.17.69.66:46493): tablet ca1440e803e946c28739eb37c13e92ab (table test-table [id=6587ef3cad4d43b799f3c231bd0f335a]) successfully deleted
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:27.666374 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:27.640915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.17.69.65:33541)
0812 06:51:27.648455 (+ 7540us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:27.648461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:27.648868 (+ 407us) server_negotiation.cc:407] Connection header received
0812 06:51:27.649041 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:27.649047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:27.649108 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:27.649215 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:27.650427 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.651515 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.652814 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.653440 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.653568 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:27.660828 (+ 7260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:27.660844 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:27.660861 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:27.660893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:27.663666 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.664315 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.664323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.664326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.664455 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.664816 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.664820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.664822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.665187 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:27.665487 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:27.665815 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 06:51:27.666027 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7397,"thread_start_us":7603,"threads_started":1}
W20260812 06:51:27.666961 21152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59988
W20260812 06:51:27.667491 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20928: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:27.667689 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20928
I20260812 06:51:27.691636 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21166
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:27.705547 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:27.692411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.17.69.66:46493)
0812 06:51:27.692906 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:27.692910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:27.692926 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:51:27.692977 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:27.692981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:27.693039 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:27.693123 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:27.694251 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.695006 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.696126 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.696513 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.696608 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:27.700326 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:27.700339 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:27.700353 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:27.700389 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:27.703425 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.703950 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.703956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.703958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.704091 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.704348 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.704351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.704353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.704688 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:27.704906 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:27.705112 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 06:51:27.705275 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":237,"threads_started":1}
W20260812 06:51:27.706176 21388 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34264
W20260812 06:51:27.706780 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21166: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:27.706902 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21166
I20260812 06:51:27.736117 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21401
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[20694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:27.749930 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:27.736796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.17.69.67:40341)
0812 06:51:27.737299 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:27.737303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:27.737319 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:51:27.737372 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:27.737376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:27.737433 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:27.737520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:27.738716 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.739448 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.740552 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:27.740919 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:27.741004 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:27.744804 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:27.744820 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:27.744836 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:27.744871 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:27.747617 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.748233 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.748241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.748243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.748384 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:27.748678 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:27.748681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:27.748683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:27.749026 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:27.749256 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:27.749433 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 06:51:27.749591 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":241,"threads_started":1}
W20260812 06:51:27.750684 21605 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56830
W20260812 06:51:27.751242 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21401: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:27.751394 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21401
I20260812 06:51:27.777410 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20719
I20260812 06:51:27.778532 20865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:27.890188 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20719
2026-08-12T06:51:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8826 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[21948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[21948](info): set up 2 sockets
Aug 12 06:51:27 dist-test-slave-6zbq krb5kdc[21948](info): commencing operation
krb5kdc: starting...
W20260812 06:51:29.112267 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:29 dist-test-slave-6zbq krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:29.289496 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38969
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:29.595808 21973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:29.596087 21973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:29.596218 21973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:29.603360 21973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:29.603466 21973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:29.603550 21973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:29.603628 21973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:29.603688 21973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:29.612970 21973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:29.614073 21973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:29.615350 21973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:29.625870 21981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:29.625913 21982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:29.625953 21984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:29.626730 21973 server_base.cc:1034] running on GCE node
I20260812 06:51:29.627445 21973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:29.628701 21973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:29.629900 21973 hybrid_clock.cc:648] HybridClock initialized: now 1786517489629854 us; error 39 us; skew 500 ppm
I20260812 06:51:29.630223 21973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:29 dist-test-slave-6zbq krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:29.633157 21973 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:29.634513 21973 webserver.cc:460] Webserver started at http://127.17.69.126:39409/ using document root <none> and password file <none>
I20260812 06:51:29.635093 21973 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:29.635198 21973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:29.635433 21973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:29.638106 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "ecceeaf3e43940d9a8dd12990e967104"
format_stamp: "Formatted at 2026-08-12 06:51:29 on dist-test-slave-6zbq"
server_key: "e1812af8fd430013a04f7d0dff7808a7"
server_key_iv: "6c6b1962bc6315979c6cd0e7b1face0f"
server_key_version: "encryptionkey@0"
I20260812 06:51:29.638837 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "ecceeaf3e43940d9a8dd12990e967104"
format_stamp: "Formatted at 2026-08-12 06:51:29 on dist-test-slave-6zbq"
server_key: "e1812af8fd430013a04f7d0dff7808a7"
server_key_iv: "6c6b1962bc6315979c6cd0e7b1face0f"
server_key_version: "encryptionkey@0"
I20260812 06:51:29.643574 21973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 06:51:29.646641 21997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:29.647787 21973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:29.647922 21973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "ecceeaf3e43940d9a8dd12990e967104"
format_stamp: "Formatted at 2026-08-12 06:51:29 on dist-test-slave-6zbq"
server_key: "e1812af8fd430013a04f7d0dff7808a7"
server_key_iv: "6c6b1962bc6315979c6cd0e7b1face0f"
server_key_version: "encryptionkey@0"
I20260812 06:51:29.648072 21973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:29.670537 21973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:29.674783 21973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:29.675009 21973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:29.691507 21973 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38969
I20260812 06:51:29.691525 22090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38969 every 8 connection(s)
I20260812 06:51:29.692725 21973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:29.693602 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21973
I20260812 06:51:29.693835 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:29.694514 17685 external_mini_cluster.cc:1428] Setting key cbab00d2d7692a398a655727d552228d
I20260812 06:51:29.696614 22091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:29.707288 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104: Bootstrap starting.
Aug 12 06:51:29 dist-test-slave-6zbq krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:29.710814 22091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:29.712074 22091 log.cc:824] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:29.715261 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104: No bootstrap required, opened a new log
I20260812 06:51:29.716635 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:29.696373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.17.69.126:38969)
0812 06:51:29.697135 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:29.697147 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:29.697188 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:51:29.698879 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:29.698903 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:29.699272 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:29.699637 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:29.701045 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:29.702400 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:29.704037 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:29.704714 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:29.704880 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:29.708657 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:29.708669 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:29.708688 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:29.708733 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:29.712419 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:29.713181 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:29.713187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:29.713193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:29.713351 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:29.713713 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:29.713718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:29.713721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:29.714403 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:29.714688 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:29.714982 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:51:29.715392 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":360,"threads_started":1}
I20260812 06:51:29.723282 22091 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } }
I20260812 06:51:29.723635 22091 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:29.723724 22091 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecceeaf3e43940d9a8dd12990e967104, State: Initialized, Role: FOLLOWER
I20260812 06:51:29.724164 22091 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } }
I20260812 06:51:29.724352 22091 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:29.724447 22091 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:29.724561 22091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:29.726552 22091 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } }
I20260812 06:51:29.726951 22091 leader_election.cc:302] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecceeaf3e43940d9a8dd12990e967104; no voters:
I20260812 06:51:29.727443 22091 leader_election.cc:288] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:29.727567 22096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:29.728143 22096 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [term 1 LEADER]: Becoming Leader. State: Replica: ecceeaf3e43940d9a8dd12990e967104, State: Running, Role: LEADER
I20260812 06:51:29.728636 22096 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } }
I20260812 06:51:29.729239 22091 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:29.731074 22097 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecceeaf3e43940d9a8dd12990e967104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } } }
I20260812 06:51:29.731460 22097 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:29.731704 22098 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecceeaf3e43940d9a8dd12990e967104. Latest consensus state: current_term: 1 leader_uuid: "ecceeaf3e43940d9a8dd12990e967104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecceeaf3e43940d9a8dd12990e967104" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38969 } } }
I20260812 06:51:29.732323 22098 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:29.733237 22106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:29.738432 22106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:29.746068 22106 catalog_manager.cc:1347] Generated new cluster ID: 2e6e0a8a98fd4339aeba339790f0107f
I20260812 06:51:29.746167 22106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:29.756469 22106 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:29.758193 22106 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:29.768622 22106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecceeaf3e43940d9a8dd12990e967104: Generated new TSK 0
I20260812 06:51:29.769507 22106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:29.821475 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:30.139437 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:30.139700 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:30.139832 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:30.146864 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:30.146965 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:30.147112 22135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:30.156276 22135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:30.157354 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:30.158555 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:30.170501 22147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:30.170542 22148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:30.170512 22153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:30.171289 22135 server_base.cc:1034] running on GCE node
I20260812 06:51:30.171818 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:30.172655 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:30.173854 22135 hybrid_clock.cc:648] HybridClock initialized: now 1786517490173810 us; error 35 us; skew 500 ppm
I20260812 06:51:30.174140 22135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:30 dist-test-slave-6zbq krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517490, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:30.177021 22135 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:30.178383 22135 webserver.cc:460] Webserver started at http://127.17.69.65:46201/ using document root <none> and password file <none>
I20260812 06:51:30.179014 22135 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:30.179131 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:30.179404 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:30.182200 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "780866110dc14b5ab66f56bd8bbf492f"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "5efb0241466e81dfc6c18be73bc97589"
server_key_iv: "804725a2faa9cbfc4cf30f59ad672306"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.182993 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "780866110dc14b5ab66f56bd8bbf492f"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "5efb0241466e81dfc6c18be73bc97589"
server_key_iv: "804725a2faa9cbfc4cf30f59ad672306"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.187669 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:30.190773 22164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.191936 22135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:30.192082 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "780866110dc14b5ab66f56bd8bbf492f"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "5efb0241466e81dfc6c18be73bc97589"
server_key_iv: "804725a2faa9cbfc4cf30f59ad672306"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.192266 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:30.207309 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:30.210808 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:30.211056 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:30.211894 22135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:30.213119 22135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:30.213196 22135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.213322 22135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:30.213376 22135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.241572 22135 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:35199
I20260812 06:51:30.241607 22385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:35199 every 8 connection(s)
I20260812 06:51:30.242753 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:30.249680 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22135
I20260812 06:51:30.249907 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:30.250739 17685 external_mini_cluster.cc:1428] Setting key 74d1286b6c44abf5eceba1cd11e35fa3
Aug 12 06:51:30 dist-test-slave-6zbq krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517490, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:30.262180 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:30.245035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:35481 (local address 127.17.69.126:38969)
0812 06:51:30.245454 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:30.245458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:30.246378 (+ 920us) server_negotiation.cc:407] Connection header received
0812 06:51:30.248060 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:30.248064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:30.248149 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:30.248243 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:30.249589 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.250745 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.252702 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.253289 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.253418 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:30.256733 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:30.256738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:30.256742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:30.256774 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:30.259227 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:30.259970 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:30.259974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:30.259976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:30.260087 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:30.260534 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:30.260539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:30.260542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:30.260818 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:30.260956 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:30.261607 (+ 651us) server_negotiation.cc:299] Negotiation successful
0812 06:51:30.261828 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":206,"threads_started":1}
I20260812 06:51:30.263041 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:30.245530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38969 (local address 127.17.69.65:35481)
0812 06:51:30.246197 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:30.246236 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:30.247835 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:30.248418 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:30.248425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:30.248880 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:30.249425 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:30.249442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.250905 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.250909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:30.252571 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:30.252587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.253481 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.253486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:30.253669 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:30.254249 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:30.254270 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:30.256522 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:30.259355 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:30.259363 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:30.259376 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:30.259877 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:30.260179 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:30.260183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:30.260186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:30.260421 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:30.260946 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:30.260954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:30.261263 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 06:51:30.261490 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 06:51:30.261943 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":247,"threads_started":1}
I20260812 06:51:30.264472 22386 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38969
I20260812 06:51:30.264748 22386 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:30.265398 22386 heartbeater.cc:511] Master 127.17.69.126:38969 requested a full tablet report, sending...
I20260812 06:51:30.267561 22031 ts_manager.cc:194] Registered new tserver with Master: 780866110dc14b5ab66f56bd8bbf492f (127.17.69.65:35199)
I20260812 06:51:30.269636 22031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:35481
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:30.295245 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:30.602784 22399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:30.603012 22399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:30.603143 22399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:30.613632 22399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:30.613799 22399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:30.613972 22399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:30.623872 22399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:30.624933 22399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:30.626121 22399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:30.638067 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:30.638209 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:30.638417 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:30.639127 22399 server_base.cc:1034] running on GCE node
I20260812 06:51:30.639673 22399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:30.640532 22399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:30.641737 22399 hybrid_clock.cc:648] HybridClock initialized: now 1786517490641676 us; error 52 us; skew 500 ppm
I20260812 06:51:30.642086 22399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:30 dist-test-slave-6zbq krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517490, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:30.646672 22399 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:30.648662 22399 webserver.cc:460] Webserver started at http://127.17.69.66:42811/ using document root <none> and password file <none>
I20260812 06:51:30.649339 22399 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:30.649461 22399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:30.649771 22399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:30.652752 22399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "7915cb3d196345f7a3f6f47a43680e03"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "0f5052ca33b149a2647dde0fb19e2dc3"
server_key_iv: "d788dc05a9780fe981b975f83228aa87"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.653504 22399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "7915cb3d196345f7a3f6f47a43680e03"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "0f5052ca33b149a2647dde0fb19e2dc3"
server_key_iv: "d788dc05a9780fe981b975f83228aa87"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.658321 22399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 06:51:30.661728 22424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.662984 22399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:51:30.663126 22399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "7915cb3d196345f7a3f6f47a43680e03"
format_stamp: "Formatted at 2026-08-12 06:51:30 on dist-test-slave-6zbq"
server_key: "0f5052ca33b149a2647dde0fb19e2dc3"
server_key_iv: "d788dc05a9780fe981b975f83228aa87"
server_key_version: "encryptionkey@0"
I20260812 06:51:30.663300 22399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:30.680826 22399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:30.684775 22399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:30.685029 22399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:30.685883 22399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:30.687156 22399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:30.687234 22399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.687374 22399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:30.687429 22399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:30.719234 22399 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:46233
I20260812 06:51:30.719296 22624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:46233 every 8 connection(s)
I20260812 06:51:30.720358 22399 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:30.722107 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22399
I20260812 06:51:30.722324 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:30.722926 17685 external_mini_cluster.cc:1428] Setting key 257a78e0199b63884e57f4259bb407e9
Aug 12 06:51:30 dist-test-slave-6zbq krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517490, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:30.740147 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:30.722685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:59643 (local address 127.17.69.126:38969)
0812 06:51:30.722839 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:30.722844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:30.724295 (+ 1451us) server_negotiation.cc:407] Connection header received
0812 06:51:30.726319 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:30.726324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:30.726428 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:30.726512 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:30.727939 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.728646 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.730230 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.730828 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.730937 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:30.734404 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:30.734412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:30.734417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:30.734455 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:30.737345 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:30.738004 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:30.738009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:30.738013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:30.738167 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:30.738584 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:30.738588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:30.738591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:30.738905 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:30.739032 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:30.739646 (+ 614us) server_negotiation.cc:299] Negotiation successful
0812 06:51:30.739851 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:51:30.740847 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:30.723277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38969 (local address 127.17.69.66:59643)
0812 06:51:30.724122 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:30.724178 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:30.726054 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:30.726763 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:30.726771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:30.727244 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:30.727777 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:30.727795 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.728798 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.728802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:30.730092 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:30.730106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:30.731055 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:30.731062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:30.731274 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:30.731855 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:30.731881 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:30.734188 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:30.737471 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:30.737478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:30.737488 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:30.737884 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:30.738260 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:30.738263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:30.738265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:30.738476 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:30.739008 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:30.739013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:30.739310 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 06:51:30.739527 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 06:51:30.739871 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":337,"threads_started":1}
I20260812 06:51:30.742112 22625 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38969
I20260812 06:51:30.742455 22625 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:30.743319 22625 heartbeater.cc:511] Master 127.17.69.126:38969 requested a full tablet report, sending...
I20260812 06:51:30.744897 22031 ts_manager.cc:194] Registered new tserver with Master: 7915cb3d196345f7a3f6f47a43680e03 (127.17.69.66:46233)
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
I20260812 06:51:30.746001 22031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:59643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:30.770991 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:31.078289 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:31.078574 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:31.078677 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:31.086360 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:31.086467 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:31.086580 22634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:31.096637 22634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:31.097721 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:31.098927 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:31.110868 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:31.111008 22644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:31.111047 22652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:31.112110 22634 server_base.cc:1034] running on GCE node
I20260812 06:51:31.112677 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:31.113536 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:31.114732 22634 hybrid_clock.cc:648] HybridClock initialized: now 1786517491114674 us; error 42 us; skew 500 ppm
I20260812 06:51:31.115123 22634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:31 dist-test-slave-6zbq krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517491, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:31.118436 22634 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:31.119725 22634 webserver.cc:460] Webserver started at http://127.17.69.67:36571/ using document root <none> and password file <none>
I20260812 06:51:31.120383 22634 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:31.120498 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:31.120785 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:31.123896 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "a58af31c6b9b46789df503bf9d451281"
format_stamp: "Formatted at 2026-08-12 06:51:31 on dist-test-slave-6zbq"
server_key: "7f9e742088a21e3fda868d7c93603813"
server_key_iv: "950d9aa9a5ee6dff25f97d2f23a754db"
server_key_version: "encryptionkey@0"
I20260812 06:51:31.124624 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "a58af31c6b9b46789df503bf9d451281"
format_stamp: "Formatted at 2026-08-12 06:51:31 on dist-test-slave-6zbq"
server_key: "7f9e742088a21e3fda868d7c93603813"
server_key_iv: "950d9aa9a5ee6dff25f97d2f23a754db"
server_key_version: "encryptionkey@0"
I20260812 06:51:31.130195 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:51:31.133363 22668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:31.134603 22634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:31.134742 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "a58af31c6b9b46789df503bf9d451281"
format_stamp: "Formatted at 2026-08-12 06:51:31 on dist-test-slave-6zbq"
server_key: "7f9e742088a21e3fda868d7c93603813"
server_key_iv: "950d9aa9a5ee6dff25f97d2f23a754db"
server_key_version: "encryptionkey@0"
I20260812 06:51:31.134905 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:31.147447 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:31.150856 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:31.151099 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:31.151958 22634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:31.153138 22634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:31.153223 22634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:31.153333 22634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:31.153388 22634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:31.184338 22634 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:45223
I20260812 06:51:31.184397 22862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:45223 every 8 connection(s)
I20260812 06:51:31.185541 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:31.186558 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22634
I20260812 06:51:31.186779 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:31.187294 17685 external_mini_cluster.cc:1428] Setting key 55b45e0aa2883415f0aca756b94a1239
Aug 12 06:51:31 dist-test-slave-6zbq krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517491, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:31.205145 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:31.188129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:50177 (local address 127.17.69.126:38969)
0812 06:51:31.188313 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:31.188317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:31.190514 (+ 2197us) server_negotiation.cc:407] Connection header received
0812 06:51:31.192356 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:31.192360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:31.192426 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:31.192508 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:31.193861 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:31.194668 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:31.196221 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:31.196695 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:31.196794 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:31.199818 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:31.199823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:31.199826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:31.199859 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:31.202755 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:31.203402 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:31.203406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:31.203408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:31.203516 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:31.203846 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:31.203849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:31.203851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:31.204099 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:31.204204 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:31.204747 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 06:51:31.204909 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:51:31.206012 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:31.189119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38969 (local address 127.17.69.67:50177)
0812 06:51:31.190366 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:31.190413 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:31.192106 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:31.192695 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:31.192702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:31.193166 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:31.193721 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:31.193739 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:31.194781 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:31.194784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:31.196070 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:31.196082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:31.196868 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:31.196873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:31.197063 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:31.197617 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:31.197638 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:31.199619 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:31.202892 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:31.202900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:31.202914 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:31.203319 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:31.203587 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:31.203590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:31.203592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:31.203769 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:31.204190 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:31.204195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:31.204471 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 06:51:31.204687 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:51:31.205013 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":708,"threads_started":1}
I20260812 06:51:31.207175 22863 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38969
I20260812 06:51:31.207438 22863 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:31.208089 22863 heartbeater.cc:511] Master 127.17.69.126:38969 requested a full tablet report, sending...
I20260812 06:51:31.209354 22031 ts_manager.cc:194] Registered new tserver with Master: a58af31c6b9b46789df503bf9d451281 (127.17.69.67:45223)
I20260812 06:51:31.210130 22031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:50177
I20260812 06:51:31.214546 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:51:31.248252 22388 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:31.246034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.17.69.126:38969)
0812 06:51:31.246253 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:31.246259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:31.246683 (+ 424us) server_negotiation.cc:407] Connection header received
0812 06:51:31.246820 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:31.246825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:31.246900 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:31.247007 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:31.248047 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58384: BlockingRecv error: recv got EOF from 127.0.0.1:58384 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 06:51:31.251163 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22135
W20260812 06:51:31.253664 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22135: Not authorized: Client connection negotiation failed: client connection to 127.17.69.65:35199: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:51:31.253822 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22135
I20260812 06:51:31.278988 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22399
W20260812 06:51:31.281315 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22399: Not authorized: Client connection negotiation failed: client connection to 127.17.69.66:46233: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:51:31.281456 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22399
I20260812 06:51:31.311439 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22634
W20260812 06:51:31.314605 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22634: Not authorized: Client connection negotiation failed: client connection to 127.17.69.67:45223: server requires authentication, but client does not have Kerberos credentials available
I20260812 06:51:31.314744 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22634
I20260812 06:51:31.336558 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21973
I20260812 06:51:31.337805 22084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:31.448450 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21973
2026-08-12T06:51:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3553 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:32 dist-test-slave-6zbq krb5kdc[22896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:32 dist-test-slave-6zbq krb5kdc[22896](info): set up 2 sockets
Aug 12 06:51:32 dist-test-slave-6zbq krb5kdc[22896](info): commencing operation
krb5kdc: starting...
W20260812 06:51:34.012312 17685 mini_kdc.cc:121] Time spent starting KDC: real 2.518s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:34.198422 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34575
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:34575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:34.503202 22917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:34.503459 22917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:34.503592 22917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:34.510355 22917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:34.510460 22917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:34.510540 22917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:34.510617 22917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:34.510694 22917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:34.519773 22917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:34575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:34.520864 22917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:34.522068 22917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:34.533114 22930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:34.533200 22931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:34.533149 22935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:34.534190 22917 server_base.cc:1034] running on GCE node
I20260812 06:51:34.535100 22917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:34.536408 22917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:34.537648 22917 hybrid_clock.cc:648] HybridClock initialized: now 1786517494537587 us; error 39 us; skew 500 ppm
I20260812 06:51:34.537998 22917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517494, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:34.545293 22917 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:34.546742 22917 webserver.cc:460] Webserver started at http://127.17.69.126:35019/ using document root <none> and password file <none>
I20260812 06:51:34.547355 22917 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:34.547472 22917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:34.547768 22917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:34.550639 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "0816e8b3df5c4235b8c35a6cbca238f2"
format_stamp: "Formatted at 2026-08-12 06:51:34 on dist-test-slave-6zbq"
server_key: "eb06dc831c3d067a425097a73e3ad46b"
server_key_iv: "cce81d3d97fdc906c4ac04d870221528"
server_key_version: "encryptionkey@0"
I20260812 06:51:34.551385 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "0816e8b3df5c4235b8c35a6cbca238f2"
format_stamp: "Formatted at 2026-08-12 06:51:34 on dist-test-slave-6zbq"
server_key: "eb06dc831c3d067a425097a73e3ad46b"
server_key_iv: "cce81d3d97fdc906c4ac04d870221528"
server_key_version: "encryptionkey@0"
I20260812 06:51:34.556067 22917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:34.559240 22954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:34.560412 22917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:51:34.560559 22917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "0816e8b3df5c4235b8c35a6cbca238f2"
format_stamp: "Formatted at 2026-08-12 06:51:34 on dist-test-slave-6zbq"
server_key: "eb06dc831c3d067a425097a73e3ad46b"
server_key_iv: "cce81d3d97fdc906c4ac04d870221528"
server_key_version: "encryptionkey@0"
I20260812 06:51:34.560743 22917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:34.576586 22917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:34.580499 22917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:34.580749 22917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:34.597870 22917 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:34575
I20260812 06:51:34.597908 23044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:34575 every 8 connection(s)
I20260812 06:51:34.599153 22917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:34.602550 23045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:34.603022 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22917
I20260812 06:51:34.603226 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:34.603864 17685 external_mini_cluster.cc:1428] Setting key c12cf6a936172c50687abd8d1410fe41
I20260812 06:51:34.618691 23045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2: Bootstrap starting.
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:34.621671 23045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:34.622761 23045 log.cc:824] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:34.625959 23045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2: No bootstrap required, opened a new log
I20260812 06:51:34.630486 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:34.606289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.17.69.126:34575)
0812 06:51:34.609185 (+ 2896us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:34.609200 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:34.609252 (+ 52us) server_negotiation.cc:407] Connection header received
0812 06:51:34.611589 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:34.611618 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:34.611999 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:34.612388 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:34.614362 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:34.616362 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:34.617581 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:34.618580 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:34.618760 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:34.622607 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:34.622624 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:34.622641 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:34.622689 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:34.626238 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:34.626932 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:34.626940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:34.626948 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:34.627139 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:34.627497 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:34.627503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:34.627506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:34.628201 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:34.628474 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:34.628825 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 06:51:34.629249 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2597,"thread_start_us":2305,"threads_started":1}
I20260812 06:51:34.634114 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:51:34.635106 23045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } }
I20260812 06:51:34.635525 23045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:34.635627 23045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0816e8b3df5c4235b8c35a6cbca238f2, State: Initialized, Role: FOLLOWER
I20260812 06:51:34.636215 23045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } }
I20260812 06:51:34.636421 23045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:34.636548 23045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:34.636690 23045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:34.638717 23045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } }
I20260812 06:51:34.639142 23045 leader_election.cc:302] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0816e8b3df5c4235b8c35a6cbca238f2; no voters:
I20260812 06:51:34.639696 23045 leader_election.cc:288] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:34.639835 23052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:34.640421 23052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [term 1 LEADER]: Becoming Leader. State: Replica: 0816e8b3df5c4235b8c35a6cbca238f2, State: Running, Role: LEADER
I20260812 06:51:34.640971 23052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } }
I20260812 06:51:34.643381 23056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } } }
I20260812 06:51:34.643791 23056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:34.644127 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0816e8b3df5c4235b8c35a6cbca238f2. Latest consensus state: current_term: 1 leader_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0816e8b3df5c4235b8c35a6cbca238f2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34575 } } }
I20260812 06:51:34.644472 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:34.645452 23045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:34.652448 23061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:34.657407 23061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:34.670157 23061 catalog_manager.cc:1347] Generated new cluster ID: 4231fe4e83e442829f08498b7e019f2d
I20260812 06:51:34.670254 23061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:34.679339 23061 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:34.680917 23061 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:34.687630 23061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0816e8b3df5c4235b8c35a6cbca238f2: Generated new TSK 0
I20260812 06:51:34.688323 23061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:51:34.692142 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:34.683438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.17.69.126:34575)
0812 06:51:34.683598 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:34.683604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:34.684223 (+ 619us) server_negotiation.cc:407] Connection header received
0812 06:51:34.684405 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:34.684411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:34.684496 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:34.684627 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:34.685796 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:34.686922 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:34.688842 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:34.689522 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:34.689620 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:34.690740 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:34.690745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:34.690748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:34.691483 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:34.691542 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:34.691569 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:34.691647 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:34.691810 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 06:51:34.691882 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:51:34.699143 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22917
I20260812 06:51:34.700685 23036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:34.785144 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22917
2026-08-12T06:51:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3329 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[23111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[23111](info): set up 2 sockets
Aug 12 06:51:34 dist-test-slave-6zbq krb5kdc[23111](info): commencing operation
krb5kdc: starting...
W20260812 06:51:36.008962 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:36 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:36.189409 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38765
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:36.494835 23136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:36.495075 23136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:36.495194 23136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:36.502007 23136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:36.502095 23136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:36.502171 23136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:36.502245 23136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:36.502305 23136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:36.511399 23136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:36.512475 23136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:36.513629 23136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:36.524498 23150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:36.524537 23151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:36.524562 23155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:36.525687 23136 server_base.cc:1034] running on GCE node
I20260812 06:51:36.526427 23136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:36.527715 23136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:36.528913 23136 hybrid_clock.cc:648] HybridClock initialized: now 1786517496528864 us; error 32 us; skew 500 ppm
I20260812 06:51:36.529218 23136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:36 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517496, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:36.532043 23136 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:36.533305 23136 webserver.cc:460] Webserver started at http://127.17.69.126:44477/ using document root <none> and password file <none>
I20260812 06:51:36.533878 23136 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:36.533974 23136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:36.534205 23136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:36.536924 23136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "8282470defb34cacbff5abc35d318271"
format_stamp: "Formatted at 2026-08-12 06:51:36 on dist-test-slave-6zbq"
server_key: "00e9c19e2b44a3d3d33b68fe75bfda22"
server_key_iv: "ec5a04b502413ccf59907a30f09951cd"
server_key_version: "encryptionkey@0"
I20260812 06:51:36.537583 23136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "8282470defb34cacbff5abc35d318271"
format_stamp: "Formatted at 2026-08-12 06:51:36 on dist-test-slave-6zbq"
server_key: "00e9c19e2b44a3d3d33b68fe75bfda22"
server_key_iv: "ec5a04b502413ccf59907a30f09951cd"
server_key_version: "encryptionkey@0"
I20260812 06:51:36.542204 23136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:51:36.545281 23164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:36.546507 23136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:36.546648 23136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "8282470defb34cacbff5abc35d318271"
format_stamp: "Formatted at 2026-08-12 06:51:36 on dist-test-slave-6zbq"
server_key: "00e9c19e2b44a3d3d33b68fe75bfda22"
server_key_iv: "ec5a04b502413ccf59907a30f09951cd"
server_key_version: "encryptionkey@0"
I20260812 06:51:36.546818 23136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:36.558207 23136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:36.561707 23136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:36.561949 23136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:36.578648 23136 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38765
I20260812 06:51:36.578692 23264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38765 every 8 connection(s)
I20260812 06:51:36.579841 23136 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:36.583278 23265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:36.584228 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23136
I20260812 06:51:36.584424 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:36.585029 17685 external_mini_cluster.cc:1428] Setting key 2ac3ebb4016e89f9f91142d45f95f008
I20260812 06:51:36.594868 23265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: Bootstrap starting.
Aug 12 06:51:36 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:36.597616 23265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:36.598613 23265 log.cc:824] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:36.601428 23265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: No bootstrap required, opened a new log
I20260812 06:51:36.606158 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:36.586888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.17.69.126:38765)
0812 06:51:36.587674 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:36.587686 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:36.587724 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:51:36.589239 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:36.589265 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:36.589625 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:36.589956 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:36.591342 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:36.592629 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:36.593847 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:36.594532 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:36.594713 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:36.598385 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:36.598395 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:36.598408 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:36.598461 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:36.601547 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:36.602158 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:36.602165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:36.602170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:36.602328 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:36.602693 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:36.602699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:36.602701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:36.603235 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:36.603546 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:36.604257 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 06:51:36.604805 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":573,"thread_start_us":340,"threads_started":1}
I20260812 06:51:36.611006 23265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } }
I20260812 06:51:36.611339 23265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:36.611428 23265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8282470defb34cacbff5abc35d318271, State: Initialized, Role: FOLLOWER
I20260812 06:51:36.611914 23265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } }
I20260812 06:51:36.612114 23265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:36.612247 23265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:36.612385 23265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:36.614521 23265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } }
I20260812 06:51:36.614979 23265 leader_election.cc:302] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8282470defb34cacbff5abc35d318271; no voters:
I20260812 06:51:36.615476 23265 leader_election.cc:288] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:36.615581 23272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:36.616242 23272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [term 1 LEADER]: Becoming Leader. State: Replica: 8282470defb34cacbff5abc35d318271, State: Running, Role: LEADER
I20260812 06:51:36.616741 23272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } }
I20260812 06:51:36.617333 23265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:36.619450 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8282470defb34cacbff5abc35d318271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } } }
I20260812 06:51:36.619477 23276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8282470defb34cacbff5abc35d318271. Latest consensus state: current_term: 1 leader_uuid: "8282470defb34cacbff5abc35d318271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8282470defb34cacbff5abc35d318271" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38765 } } }
I20260812 06:51:36.619902 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:36.619902 23276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271 [sys.catalog]: This master's current role is: LEADER
W20260812 06:51:36.631940 23304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:51:36.632038 23304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:51:36.632158 23287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:36.634220 23287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:36.642156 23287 catalog_manager.cc:1347] Generated new cluster ID: 957d2904b9e048a49b2459ec295139f7
I20260812 06:51:36.642246 23287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:36.665230 23287 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:36.666549 23287 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:36.672240 23287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8282470defb34cacbff5abc35d318271: Generated new TSK 0
I20260812 06:51:36.672902 23287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:36.723222 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:37.033008 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:37.033269 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:37.033383 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:37.040179 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:37.040283 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:37.040433 23314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:37.049742 23314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:37.050995 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:37.052186 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:37.064013 23322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:37.064098 23321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:37.064110 23325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:37.064970 23314 server_base.cc:1034] running on GCE node
I20260812 06:51:37.065505 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:37.066290 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:37.067500 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786517497067453 us; error 40 us; skew 500 ppm
I20260812 06:51:37.067785 23314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:37 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:37.070776 23314 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:37.072013 23314 webserver.cc:460] Webserver started at http://127.17.69.65:37803/ using document root <none> and password file <none>
I20260812 06:51:37.072628 23314 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:37.072742 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:37.073021 23314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:37.075721 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "6279bd08759c44c791e4800516918ae4"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "ca13e26bcd283bc382e0ee743270215e"
server_key_iv: "59b7aff556f9dcc3747699868ea1c987"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.076550 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "6279bd08759c44c791e4800516918ae4"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "ca13e26bcd283bc382e0ee743270215e"
server_key_iv: "59b7aff556f9dcc3747699868ea1c987"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.081284 23314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 06:51:37.084383 23335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.085525 23314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:51:37.085671 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "6279bd08759c44c791e4800516918ae4"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "ca13e26bcd283bc382e0ee743270215e"
server_key_iv: "59b7aff556f9dcc3747699868ea1c987"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.085835 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:37.120512 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:37.124644 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:37.124910 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:37.125826 23314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:37.127154 23314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:37.127243 23314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.127368 23314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:37.127421 23314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.157982 23314 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:34867
I20260812 06:51:37.158038 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:34867 every 8 connection(s)
I20260812 06:51:37.159224 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:37.160668 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23314
I20260812 06:51:37.160861 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:37.161712 17685 external_mini_cluster.cc:1428] Setting key e039c841e70211e9a8cac45e185a0b74
Aug 12 06:51:37 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:37.182183 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:37.161587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:59919 (local address 127.17.69.126:38765)
0812 06:51:37.162688 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:37.162695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:37.166934 (+ 4239us) server_negotiation.cc:407] Connection header received
0812 06:51:37.168659 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:37.168665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:37.168739 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:37.168841 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:37.170275 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.171035 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.172565 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.172970 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.173079 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:37.176678 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:37.176683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:37.176687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:37.176718 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:37.179069 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:37.179903 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:37.179907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:37.179909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:37.180022 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:37.180475 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:37.180478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:37.180480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:37.180735 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:37.180848 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:37.181650 (+ 802us) server_negotiation.cc:299] Negotiation successful
0812 06:51:37.181878 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":849,"threads_started":1}
I20260812 06:51:37.183461 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:37.162227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38765 (local address 127.17.69.65:59919)
0812 06:51:37.166791 (+ 4564us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:37.166835 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:37.168419 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:37.169067 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:37.169075 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:37.169563 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:37.170112 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:37.170129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.171174 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.171177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:37.172443 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:37.172458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.173183 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.173191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:37.173409 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:37.174112 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:37.174137 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:37.176458 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:37.179207 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:37.179215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:37.179229 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:37.179800 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:37.180124 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:37.180128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:37.180132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:37.180375 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:37.180856 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:37.180864 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:37.181294 (+ 430us) client_negotiation.cc:769] Sending connection context
0812 06:51:37.181594 (+ 300us) client_negotiation.cc:240] Negotiation successful
0812 06:51:37.182108 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4338,"thread_start_us":4126,"threads_started":1}
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
I20260812 06:51:37.184855 23553 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38765
I20260812 06:51:37.185163 23553 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:37.185824 23553 heartbeater.cc:511] Master 127.17.69.126:38765 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
I20260812 06:51:37.187892 23190 ts_manager.cc:194] Registered new tserver with Master: 6279bd08759c44c791e4800516918ae4 (127.17.69.65:34867)
I20260812 06:51:37.189319 23190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:59919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:37.209765 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:37.520882 23569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:37.521137 23569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:37.521271 23569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:37.528297 23569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:37.528393 23569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:37.528546 23569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:37.537859 23569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:37.538990 23569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:37.540217 23569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:37.552497 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:37.552697 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:37.553102 23587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:37.553484 23569 server_base.cc:1034] running on GCE node
I20260812 06:51:37.554023 23569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:37.554914 23569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:37.556110 23569 hybrid_clock.cc:648] HybridClock initialized: now 1786517497556058 us; error 38 us; skew 500 ppm
I20260812 06:51:37.556391 23569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:37 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:37.559369 23569 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:37.560685 23569 webserver.cc:460] Webserver started at http://127.17.69.66:35033/ using document root <none> and password file <none>
I20260812 06:51:37.561296 23569 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:37.561405 23569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:37.561687 23569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:37.564437 23569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "3141704d78e4403095454f6c5c93a848"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "72c86bd06807d16f59c65ea900c66e1a"
server_key_iv: "663a7139f2d1ec44b3b2b48e5b1f3f19"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.565166 23569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "3141704d78e4403095454f6c5c93a848"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "72c86bd06807d16f59c65ea900c66e1a"
server_key_iv: "663a7139f2d1ec44b3b2b48e5b1f3f19"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.569921 23569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 06:51:37.573038 23597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.574189 23569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:37.574386 23569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "3141704d78e4403095454f6c5c93a848"
format_stamp: "Formatted at 2026-08-12 06:51:37 on dist-test-slave-6zbq"
server_key: "72c86bd06807d16f59c65ea900c66e1a"
server_key_iv: "663a7139f2d1ec44b3b2b48e5b1f3f19"
server_key_version: "encryptionkey@0"
I20260812 06:51:37.574572 23569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:37.585906 23569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:37.589632 23569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:37.589872 23569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:37.590773 23569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:37.591967 23569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:37.592036 23569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.592139 23569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:37.592190 23569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:37.621192 23569 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:37003
I20260812 06:51:37.621249 23818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:37003 every 8 connection(s)
I20260812 06:51:37.622295 23569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:37.626717 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23569
I20260812 06:51:37.626876 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:37.627310 17685 external_mini_cluster.cc:1428] Setting key 58e241fa422dfb4573ec74832aec4430
Aug 12 06:51:37 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517497, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:37.641748 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:37.624678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:43403 (local address 127.17.69.126:38765)
0812 06:51:37.624881 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:37.624888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:37.625874 (+ 986us) server_negotiation.cc:407] Connection header received
0812 06:51:37.627872 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:37.627878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:37.627965 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:37.628049 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:37.629591 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.630574 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.632028 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.632566 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.632682 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:37.636007 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:37.636013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:37.636016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:37.636048 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:37.638881 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:37.639516 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:37.639522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:37.639525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:37.639689 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:37.640133 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:37.640137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:37.640141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:37.640489 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:37.640654 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:37.641235 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 06:51:37.641453 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:51:37.642438 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:37.625081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38765 (local address 127.17.69.66:43403)
0812 06:51:37.625732 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:37.625776 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:37.627620 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:37.628295 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:37.628303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:37.628805 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:37.629404 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:37.629423 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.630721 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.630725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:37.631888 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:37.631902 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:37.632752 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:37.632756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:37.632944 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:37.633506 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:37.633528 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:37.635803 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:37.639005 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:37.639010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:37.639020 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:37.639408 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:37.639825 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:37.639829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:37.639832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:37.640014 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:37.640605 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:37.640609 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:37.640874 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 06:51:37.641109 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 06:51:37.641449 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":248,"threads_started":1}
I20260812 06:51:37.643646 23819 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38765
I20260812 06:51:37.643935 23819 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:37.644668 23819 heartbeater.cc:511] Master 127.17.69.126:38765 requested a full tablet report, sending...
I20260812 06:51:37.646142 23190 ts_manager.cc:194] Registered new tserver with Master: 3141704d78e4403095454f6c5c93a848 (127.17.69.66:37003)
I20260812 06:51:37.646979 23190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:43403
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:37.674393 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:37.980939 23827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:37.981189 23827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:37.981323 23827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:37.988142 23827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:37.988240 23827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:37.988389 23827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:37.997869 23827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:37.998947 23827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:38.000136 23827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:38.011751 23839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:38.011860 23842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:38.011883 23847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:38.012849 23827 server_base.cc:1034] running on GCE node
I20260812 06:51:38.013353 23827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:38.014115 23827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:38.015305 23827 hybrid_clock.cc:648] HybridClock initialized: now 1786517498015246 us; error 50 us; skew 500 ppm
I20260812 06:51:38.015556 23827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:38.018421 23827 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:38.019770 23827 webserver.cc:460] Webserver started at http://127.17.69.67:44447/ using document root <none> and password file <none>
I20260812 06:51:38.020364 23827 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:38.020478 23827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:38.020764 23827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:38.023486 23827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "f36f2ac9ca0044909d9f3adde2093c4a"
format_stamp: "Formatted at 2026-08-12 06:51:38 on dist-test-slave-6zbq"
server_key: "f2197ecb733115a7d576cd33384e87ef"
server_key_iv: "1f36ed767a2c83c6d46419a242624424"
server_key_version: "encryptionkey@0"
I20260812 06:51:38.024246 23827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "f36f2ac9ca0044909d9f3adde2093c4a"
format_stamp: "Formatted at 2026-08-12 06:51:38 on dist-test-slave-6zbq"
server_key: "f2197ecb733115a7d576cd33384e87ef"
server_key_iv: "1f36ed767a2c83c6d46419a242624424"
server_key_version: "encryptionkey@0"
I20260812 06:51:38.029026 23827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:38.032236 23862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:38.033380 23827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:38.033520 23827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "f36f2ac9ca0044909d9f3adde2093c4a"
format_stamp: "Formatted at 2026-08-12 06:51:38 on dist-test-slave-6zbq"
server_key: "f2197ecb733115a7d576cd33384e87ef"
server_key_iv: "1f36ed767a2c83c6d46419a242624424"
server_key_version: "encryptionkey@0"
I20260812 06:51:38.033690 23827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:38.047819 23827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:38.051502 23827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:38.051757 23827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:38.052594 23827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:38.053800 23827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:38.053884 23827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:38.053999 23827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:38.054054 23827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:38.083547 23827 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:40807
I20260812 06:51:38.083601 24069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:40807 every 8 connection(s)
I20260812 06:51:38.084666 23827 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:38.089634 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23827
I20260812 06:51:38.089799 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:38.090420 17685 external_mini_cluster.cc:1428] Setting key d83354e1591b3f8dff5ce7191264adc5
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:38.102284 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.086890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45205 (local address 127.17.69.126:38765)
0812 06:51:38.087081 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:38.087088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:38.088143 (+ 1055us) server_negotiation.cc:407] Connection header received
0812 06:51:38.089913 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:38.089919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:38.090005 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:38.090111 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:38.091698 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.092642 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.094272 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.094712 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.094805 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.097747 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:38.097752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:38.097755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:38.097783 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:38.100039 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.100613 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.100617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.100619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.100735 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.101049 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.101052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.101054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.101306 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:38.101411 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:38.101903 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 06:51:38.102052 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:51:38.103101 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.087342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38765 (local address 127.17.69.67:45205)
0812 06:51:38.087997 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:38.088038 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:38.089647 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:38.090308 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:38.090315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:38.090840 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:38.091523 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:38.091548 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.092810 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.092816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:38.094151 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:38.094163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.094879 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.094884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:38.095065 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.095613 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:38.095634 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:38.097583 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:38.100157 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:38.100163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:38.100173 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:38.100532 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:38.100811 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:38.100815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:38.100817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:38.100975 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:38.101390 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:38.101395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:38.101640 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 06:51:38.101846 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 06:51:38.102167 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":256,"threads_started":1}
I20260812 06:51:38.104357 24070 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38765
I20260812 06:51:38.104595 24070 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:38.105223 24070 heartbeater.cc:511] Master 127.17.69.126:38765 requested a full tablet report, sending...
I20260812 06:51:38.106539 23190 ts_manager.cc:194] Registered new tserver with Master: f36f2ac9ca0044909d9f3adde2093c4a (127.17.69.67:40807)
I20260812 06:51:38.107300 23190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:45205
I20260812 06:51:38.116820 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:38.167021 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.153858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.17.69.126:38765)
0812 06:51:38.154032 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:38.154036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:38.154544 (+ 508us) server_negotiation.cc:407] Connection header received
0812 06:51:38.154718 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:38.154722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:38.154787 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:38.154862 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:38.156167 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.156880 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.158121 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.158776 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.158920 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.162628 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:38.162634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:38.162638 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:38.162677 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:38.164909 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.165455 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.165458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.165460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.165581 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.165833 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.165836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.165838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.166130 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:38.166298 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:38.166594 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 06:51:38.166746 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 06:51:38.167529 23190 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38144
I20260812 06:51:38.169067 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23314
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:38.182436 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.169872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.17.69.65:34867)
0812 06:51:38.170404 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:38.170413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:38.170438 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:51:38.170486 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:38.170491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:38.170671 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:38.170782 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:38.171741 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.172690 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.173775 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.174287 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.174457 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.177804 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:38.177815 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:38.177826 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:38.177864 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:38.180319 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.180908 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.180913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.180916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.181025 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.181298 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.181301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.181303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.181581 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:38.181798 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:38.181966 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 06:51:38.182135 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":238,"threads_started":1}
W20260812 06:51:38.183490 23544 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55404
W20260812 06:51:38.184420 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23314: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:38.184587 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23314
I20260812 06:51:38.203770 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23569
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:38.217779 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.204434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.17.69.66:37003)
0812 06:51:38.204901 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:38.204907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:38.204930 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:51:38.204985 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:38.204991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:38.205195 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:38.205310 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:38.206418 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.207380 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.208575 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.209095 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.209219 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.212903 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:38.212915 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:38.212926 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:38.212970 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:38.215450 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.216012 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.216017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.216019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.216153 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.216435 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.216438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.216440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.216748 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:38.216986 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:38.217246 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 06:51:38.217451 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":224,"threads_started":1}
W20260812 06:51:38.218983 23811 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37994
W20260812 06:51:38.219906 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23569: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:38.220028 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23569
I20260812 06:51:38.241569 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23827
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[23111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517498, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:38.256716 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:38.242501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.17.69.67:40807)
0812 06:51:38.243154 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:38.243162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:38.243194 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:51:38.243255 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:38.243261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:38.243465 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:38.243590 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:38.244919 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.245956 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.247252 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:38.247789 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:38.247885 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:38.251742 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:38.251754 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:38.251768 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:38.251800 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:38.254424 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.254979 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.254985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.254987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.255101 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:38.255428 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:38.255431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:38.255433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:38.255752 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:38.255971 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:38.256181 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 06:51:38.256366 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":332,"threads_started":1}
W20260812 06:51:38.257997 24060 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37252
W20260812 06:51:38.259019 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23827: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:51:38.259199 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23827
I20260812 06:51:38.281021 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23136
I20260812 06:51:38.282092 23256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:38.381791 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23136
2026-08-12T06:51:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3600 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[24095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[24095](info): set up 2 sockets
Aug 12 06:51:38 dist-test-slave-6zbq krb5kdc[24095](info): commencing operation
krb5kdc: starting...
W20260812 06:51:39.591310 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:39 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:39.787262 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:35683
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:35683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:40.096645 24123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:40.096902 24123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:40.097028 24123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:40.103936 24123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:40.104024 24123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:40.104094 24123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:40.104167 24123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:40.104225 24123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:40.113366 24123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:35683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:35683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:40.114478 24123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:40.115639 24123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:40.126410 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:40.126448 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:40.126473 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:40.127511 24123 server_base.cc:1034] running on GCE node
I20260812 06:51:40.128181 24123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:40.129380 24123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:40.130574 24123 hybrid_clock.cc:648] HybridClock initialized: now 1786517500130529 us; error 33 us; skew 500 ppm
I20260812 06:51:40.130892 24123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:40 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517500, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:40.137940 24123 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:40.139253 24123 webserver.cc:460] Webserver started at http://127.17.69.126:44381/ using document root <none> and password file <none>
I20260812 06:51:40.139855 24123 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:40.139961 24123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:40.140239 24123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:40.143049 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "9698e0c7ae9c4987a30bcda48422c982"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "599dfcdb897f3ea11971496b478260f6"
server_key_iv: "36690818866f090cdba40e09463afa74"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.143775 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "9698e0c7ae9c4987a30bcda48422c982"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "599dfcdb897f3ea11971496b478260f6"
server_key_iv: "36690818866f090cdba40e09463afa74"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.148346 24123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:40.151424 24149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:40.152546 24123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:40.152701 24123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "9698e0c7ae9c4987a30bcda48422c982"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "599dfcdb897f3ea11971496b478260f6"
server_key_iv: "36690818866f090cdba40e09463afa74"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.152872 24123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:40.178612 24123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:40.182204 24123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:40.182463 24123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:40.198732 24123 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:35683
I20260812 06:51:40.198786 24227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:35683 every 8 connection(s)
I20260812 06:51:40.199983 24123 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:40.203434 24228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:40.203568 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24123
I20260812 06:51:40.203794 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:40.204604 17685 external_mini_cluster.cc:1428] Setting key 73b7d6f1a355148b335b63416da84adc
I20260812 06:51:40.216388 24228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982: Bootstrap starting.
Aug 12 06:51:40 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:40.219919 24228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:40.220901 24228 log.cc:824] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:40.223796 24228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982: No bootstrap required, opened a new log
I20260812 06:51:40.226847 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:40.206596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.17.69.126:35683)
0812 06:51:40.207326 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:40.207337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:40.207373 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:51:40.209320 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:40.209341 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:40.209684 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:40.209980 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:40.211777 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.213091 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.214810 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.215448 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.215575 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:40.220046 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:40.220055 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:40.220068 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:40.220106 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:40.223092 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:40.223848 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:40.223856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:40.223863 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:40.224027 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:40.224428 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:40.224431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:40.224433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:40.224892 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:40.225090 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:40.225546 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 06:51:40.225880 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":534,"thread_start_us":311,"threads_started":1}
I20260812 06:51:40.231374 24228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } }
I20260812 06:51:40.231686 24228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:40.231786 24228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9698e0c7ae9c4987a30bcda48422c982, State: Initialized, Role: FOLLOWER
I20260812 06:51:40.232280 24228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } }
I20260812 06:51:40.232488 24228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:40.232610 24228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:40.232761 24228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:40.234983 24228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } }
I20260812 06:51:40.235414 24228 leader_election.cc:302] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9698e0c7ae9c4987a30bcda48422c982; no voters:
I20260812 06:51:40.235918 24228 leader_election.cc:288] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:40.236033 24234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:40.236615 24234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [term 1 LEADER]: Becoming Leader. State: Replica: 9698e0c7ae9c4987a30bcda48422c982, State: Running, Role: LEADER
I20260812 06:51:40.237133 24234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } }
I20260812 06:51:40.237670 24228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:40.240041 24238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9698e0c7ae9c4987a30bcda48422c982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } } }
I20260812 06:51:40.240423 24238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:40.241093 24239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9698e0c7ae9c4987a30bcda48422c982. Latest consensus state: current_term: 1 leader_uuid: "9698e0c7ae9c4987a30bcda48422c982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9698e0c7ae9c4987a30bcda48422c982" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35683 } } }
I20260812 06:51:40.241495 24239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:40.241695 24252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:40.246402 24252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:40.256587 24252 catalog_manager.cc:1347] Generated new cluster ID: 766c67fb00bd40f6b0ca718238f9b73e
I20260812 06:51:40.256696 24252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:40.272112 24252 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:40.273397 24252 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:40.287993 24252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9698e0c7ae9c4987a30bcda48422c982: Generated new TSK 0
I20260812 06:51:40.288735 24252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:40.341044 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:40.655301 24288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:40.655543 24288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:40.655668 24288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:40.662709 24288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:40.662817 24288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:40.662962 24288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:40.672158 24288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:40.673274 24288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:40.674516 24288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:40.686802 24306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:40.686903 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:40.686823 24304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:40.687958 24288 server_base.cc:1034] running on GCE node
I20260812 06:51:40.688541 24288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:40.689416 24288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:40.690604 24288 hybrid_clock.cc:648] HybridClock initialized: now 1786517500690567 us; error 28 us; skew 500 ppm
I20260812 06:51:40.690881 24288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:40 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517500, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:40.693807 24288 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:40.695132 24288 webserver.cc:460] Webserver started at http://127.17.69.65:38695/ using document root <none> and password file <none>
I20260812 06:51:40.695753 24288 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:40.695868 24288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:40.696146 24288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:40.698882 24288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "deb0647116ab42e593494c9075a87375"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "7edd03b38facc7038c56553664797a79"
server_key_iv: "616717fe7530698755ef72add073415c"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.699615 24288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "deb0647116ab42e593494c9075a87375"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "7edd03b38facc7038c56553664797a79"
server_key_iv: "616717fe7530698755ef72add073415c"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.704244 24288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:40.707340 24319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:40.708477 24288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:40.708624 24288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "deb0647116ab42e593494c9075a87375"
format_stamp: "Formatted at 2026-08-12 06:51:40 on dist-test-slave-6zbq"
server_key: "7edd03b38facc7038c56553664797a79"
server_key_iv: "616717fe7530698755ef72add073415c"
server_key_version: "encryptionkey@0"
I20260812 06:51:40.708806 24288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:40.727568 24288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:40.732366 24288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:40.732693 24288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:40.737208 24288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:40.738704 24288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:40.738799 24288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:40.738955 24288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:40.739042 24288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:40.767216 24288 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:37181
I20260812 06:51:40.767294 24525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:37181 every 8 connection(s)
I20260812 06:51:40.768363 24288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:40.777060 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24288
I20260812 06:51:40.777257 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:40.777873 17685 external_mini_cluster.cc:1428] Setting key 54f72999a586ed29a67c7f1c4e535053
Aug 12 06:51:40 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517500, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:40.786422 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:40.770597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:34109 (local address 127.17.69.126:35683)
0812 06:51:40.771013 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:40.771018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:40.771853 (+ 835us) server_negotiation.cc:407] Connection header received
0812 06:51:40.773609 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:40.773615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:40.773690 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:40.773791 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:40.775227 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.775976 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.777476 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.777887 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.777994 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:40.781413 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:40.781419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:40.781422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:40.781452 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:40.783690 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:40.784336 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:40.784340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:40.784343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:40.784453 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:40.784838 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:40.784842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:40.784844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:40.785094 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:40.785221 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:40.785829 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 06:51:40.786037 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":223,"threads_started":1}
I20260812 06:51:40.787263 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:40.771036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:35683 (local address 127.17.69.65:34109)
0812 06:51:40.771704 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:40.771746 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:40.773361 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:40.774017 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:40.774024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:40.774547 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:40.775062 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:40.775079 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.776124 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.776128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:40.777342 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:40.777358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:40.778101 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:40.778108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:40.778329 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:40.778959 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:40.778985 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:40.781214 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:40.783820 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:40.783829 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:40.783842 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:40.784213 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:40.784551 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:40.784556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:40.784560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:40.784726 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:40.785213 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:40.785220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:40.785500 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 06:51:40.785710 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 06:51:40.786169 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":263,"threads_started":1}
I20260812 06:51:40.788677 24526 heartbeater.cc:347] Connected to a master server at 127.17.69.126:35683
I20260812 06:51:40.788937 24526 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:40.789574 24526 heartbeater.cc:511] Master 127.17.69.126:35683 requested a full tablet report, sending...
I20260812 06:51:40.791498 24176 ts_manager.cc:194] Registered new tserver with Master: deb0647116ab42e593494c9075a87375 (127.17.69.65:37181)
I20260812 06:51:40.792850 24176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:34109
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:40.822744 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:41.131748 24537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:41.132017 24537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:41.132160 24537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:41.139246 24537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:41.139349 24537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:41.139492 24537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:41.148797 24537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:41.149940 24537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:41.151175 24537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:41.163941 24551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:41.163975 24553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:41.164472 24550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:41.164770 24537 server_base.cc:1034] running on GCE node
I20260812 06:51:41.165277 24537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:41.166088 24537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:41.167273 24537 hybrid_clock.cc:648] HybridClock initialized: now 1786517501167232 us; error 32 us; skew 500 ppm
I20260812 06:51:41.167584 24537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:41.170428 24537 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:41.171710 24537 webserver.cc:460] Webserver started at http://127.17.69.66:39347/ using document root <none> and password file <none>
I20260812 06:51:41.172294 24537 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:41.172405 24537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:41.172690 24537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:41.175390 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "7b0b2b11273247519abe6dbb3d5a272b"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "14e80706c019f3f505ee596c2c33b40d"
server_key_iv: "097567eca59d4d2059f2a6c4b0b152c9"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.176105 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "7b0b2b11273247519abe6dbb3d5a272b"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "14e80706c019f3f505ee596c2c33b40d"
server_key_iv: "097567eca59d4d2059f2a6c4b0b152c9"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.180704 24537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:41.184032 24566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.185185 24537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:51:41.185326 24537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "7b0b2b11273247519abe6dbb3d5a272b"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "14e80706c019f3f505ee596c2c33b40d"
server_key_iv: "097567eca59d4d2059f2a6c4b0b152c9"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.185511 24537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:41.211714 24537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:41.215480 24537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:41.215710 24537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:41.216528 24537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:41.217730 24537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:41.217803 24537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.217890 24537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:41.217932 24537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.247740 24537 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:41103
I20260812 06:51:41.247786 24751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:41103 every 8 connection(s)
I20260812 06:51:41.248955 24537 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:41.259505 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24537
I20260812 06:51:41.259773 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:41.260409 17685 external_mini_cluster.cc:1428] Setting key 3ec22d2cea33d9df2fc4734606199e27
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:41.277539 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.251369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:56125 (local address 127.17.69.126:35683)
0812 06:51:41.251589 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:41.251596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:41.252720 (+ 1124us) server_negotiation.cc:407] Connection header received
0812 06:51:41.254529 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:41.254534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:41.254609 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:41.254699 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:41.256369 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.257196 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.259546 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.260236 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.260389 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.265242 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:41.265251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:41.265255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:41.265319 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:41.271869 (+ 6550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.272895 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.272900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.272903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.273057 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.273636 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.273642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.273646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.274103 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:41.274307 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:41.276456 (+ 2149us) server_negotiation.cc:299] Negotiation successful
0812 06:51:41.277109 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:51:41.278182 24756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.251865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:35683 (local address 127.17.69.66:56125)
0812 06:51:41.252554 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:41.252596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:41.254216 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:41.254881 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:41.254889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:41.255401 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:41.256104 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:41.256125 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.257518 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.257530 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:41.259211 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:41.259234 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.260596 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.260602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:41.260837 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.261956 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:41.261985 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:41.264764 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:41.272039 (+ 7275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:41.272049 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:41.272068 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:41.272683 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:41.273172 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:41.273178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:41.273182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:41.273482 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:41.274393 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:41.274405 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:41.275112 (+ 707us) client_negotiation.cc:769] Sending connection context
0812 06:51:41.275728 (+ 616us) client_negotiation.cc:240] Negotiation successful
0812 06:51:41.276628 (+ 900us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":262,"threads_started":1}
I20260812 06:51:41.280788 24754 heartbeater.cc:347] Connected to a master server at 127.17.69.126:35683
I20260812 06:51:41.281296 24754 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:41.282586 24754 heartbeater.cc:511] Master 127.17.69.126:35683 requested a full tablet report, sending...
I20260812 06:51:41.285967 24176 ts_manager.cc:194] Registered new tserver with Master: 7b0b2b11273247519abe6dbb3d5a272b (127.17.69.66:41103)
I20260812 06:51:41.286847 24176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:56125
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:41.331946 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:41.695161 24760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:41.695421 24760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:41.695537 24760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:41.702307 24760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:41.702445 24760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:41.702598 24760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:41.711539 24760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:35683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:41.712628 24760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:41.713814 24760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:41.725606 24775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:41.725692 24768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:41.725687 24771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:41.726708 24760 server_base.cc:1034] running on GCE node
I20260812 06:51:41.727192 24760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:41.727967 24760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:41.729142 24760 hybrid_clock.cc:648] HybridClock initialized: now 1786517501729088 us; error 48 us; skew 500 ppm
I20260812 06:51:41.729379 24760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:41.732275 24760 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:41.733507 24760 webserver.cc:460] Webserver started at http://127.17.69.67:35835/ using document root <none> and password file <none>
I20260812 06:51:41.734087 24760 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:41.734208 24760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:41.734516 24760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:41.737239 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "d734cd1a4fcf49edb8e83890f82499bb"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "81bd5148e25a099738a4b0204aea538d"
server_key_iv: "bd63e0bc04762255a9144a51d2476452"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.737995 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "d734cd1a4fcf49edb8e83890f82499bb"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "81bd5148e25a099738a4b0204aea538d"
server_key_iv: "bd63e0bc04762255a9144a51d2476452"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.742805 24760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 06:51:41.745880 24784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.747104 24760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:41.747256 24760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "d734cd1a4fcf49edb8e83890f82499bb"
format_stamp: "Formatted at 2026-08-12 06:51:41 on dist-test-slave-6zbq"
server_key: "81bd5148e25a099738a4b0204aea538d"
server_key_iv: "bd63e0bc04762255a9144a51d2476452"
server_key_version: "encryptionkey@0"
I20260812 06:51:41.747454 24760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:41.767390 24760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:41.772158 24760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:41.772471 24760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:41.773442 24760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:41.774700 24760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:41.774775 24760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.774857 24760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:41.774900 24760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:41.795616 24526 heartbeater.cc:503] Master 127.17.69.126:35683 was elected leader, sending a full tablet report...
I20260812 06:51:41.804955 24760 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41529
I20260812 06:51:41.804991 24975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41529 every 8 connection(s)
I20260812 06:51:41.806172 24760 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:41.811892 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24760
I20260812 06:51:41.812098 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:41.812659 17685 external_mini_cluster.cc:1428] Setting key ab977b62c87023bd128e9a0a60c079a7
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:41.825590 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.808554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:56661 (local address 127.17.69.126:35683)
0812 06:51:41.808952 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:41.808956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:41.809914 (+ 958us) server_negotiation.cc:407] Connection header received
0812 06:51:41.811835 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:41.811841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:41.811916 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:41.812022 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:41.813984 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.814933 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.816562 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.816970 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.817061 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.820142 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:41.820147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:41.820150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:41.820185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:41.822485 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.823220 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.823224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.823226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.823357 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.824066 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.824072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.824076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.824399 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:41.824528 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:41.825136 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 06:51:41.825319 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":206,"threads_started":1}
I20260812 06:51:41.826669 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.809084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:35683 (local address 127.17.69.67:56661)
0812 06:51:41.809765 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:41.809801 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:41.811548 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:41.812355 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:41.812367 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:41.813119 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:41.813808 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:41.813828 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.815066 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.815070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:41.816437 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:41.816449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.817132 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.817137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:41.817343 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.817904 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:41.817928 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:41.819950 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:41.822639 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:41.822646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:41.822659 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:41.823082 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:41.823736 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:41.823740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:41.823742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:41.823934 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:41.824538 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:41.824545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:41.824851 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 06:51:41.825075 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 06:51:41.825460 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":257,"threads_started":1}
I20260812 06:51:41.827982 24978 heartbeater.cc:347] Connected to a master server at 127.17.69.126:35683
I20260812 06:51:41.828255 24978 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:41.828940 24978 heartbeater.cc:511] Master 127.17.69.126:35683 requested a full tablet report, sending...
I20260812 06:51:41.830400 24175 ts_manager.cc:194] Registered new tserver with Master: d734cd1a4fcf49edb8e83890f82499bb (127.17.69.67:41529)
I20260812 06:51:41.831244 24175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:56661
I20260812 06:51:41.839870 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:41.894577 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.880005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.17.69.65:37181)
0812 06:51:41.880649 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:41.880656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:41.880683 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:51:41.880744 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:41.880752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:41.880989 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:41.881147 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:41.882677 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.883706 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.884905 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.885456 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.885570 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.889470 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:41.889482 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:41.889496 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:41.889530 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:41.892275 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.892869 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.892875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.892877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.893000 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.893276 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.893279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.893281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.893608 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:41.893832 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:41.894038 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 06:51:41.894231 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":307,"threads_started":1}
Aug 12 06:51:41 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:41.910668 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.898695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.17.69.126:35683)
0812 06:51:41.898910 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:41.898917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:41.899043 (+ 126us) server_negotiation.cc:407] Connection header received
0812 06:51:41.899231 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:41.899236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:41.899315 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:41.899396 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:41.900575 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.901320 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.902592 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.902991 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.903100 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.906503 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:41.906508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:41.906511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:41.906541 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:41.908690 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.909205 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.909209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.909211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.909313 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.909574 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.909577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.909579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.909876 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:41.910046 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:41.910240 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 06:51:41.910448 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 06:51:41.911211 24175 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51854
I20260812 06:51:41.912106 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24288
I20260812 06:51:41.923151 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:41.912891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.17.69.65:37181)
0812 06:51:41.913044 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:41.913050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:41.913286 (+ 236us) server_negotiation.cc:407] Connection header received
0812 06:51:41.913373 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:41.913377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:41.913464 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:41.913540 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:41.914663 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.915312 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.916382 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:41.916778 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:41.916883 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:41.918795 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:41.918800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:41.918803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:41.918829 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:41.921107 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.921621 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.921624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.921626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.921742 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:41.922004 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:41.922007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:41.922009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:41.922218 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:41.922436 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:41.922698 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 06:51:41.922863 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:51:41.923800 24516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:42.006274 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24288
I20260812 06:51:42.028376 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24537
Aug 12 06:51:42 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:42.043939 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:42.029202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.17.69.66:41103)
0812 06:51:42.029733 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:42.029740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:42.029768 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:51:42.029831 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:42.029838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:42.030043 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:42.030180 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:42.031306 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:42.032299 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:42.033471 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:42.033983 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:42.034075 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:42.038292 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:42.038307 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:42.038321 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:42.038405 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:42.041213 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:42.041807 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:42.041814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:42.041817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:42.041945 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:42.042303 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:42.042307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:42.042309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:42.042818 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:42.043049 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:42.043427 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 06:51:42.043651 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":263,"threads_started":1}
I20260812 06:51:42.045233 24744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:42.125500 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24537
I20260812 06:51:42.147903 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24760
Aug 12 06:51:42 dist-test-slave-6zbq krb5kdc[24095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:42.164376 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:42.148818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.17.69.67:41529)
0812 06:51:42.149425 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:42.149432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:42.149462 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:51:42.149524 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:42.149530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:42.149745 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:42.149906 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:42.151296 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:42.152372 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:42.153627 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:42.154167 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:42.154281 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:42.158419 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:42.158430 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:42.158443 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:42.158479 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:42.161362 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:42.162084 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:42.162093 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:42.162097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:42.162293 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:42.162754 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:42.162759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:42.162763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:42.163223 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:42.163545 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:42.163812 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 06:51:42.164023 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":307,"threads_started":1}
I20260812 06:51:42.165621 24968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:42.245754 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24760
I20260812 06:51:42.269749 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24123
I20260812 06:51:42.271163 24219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:42.375720 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24123
2026-08-12T06:51:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4002 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:42 dist-test-slave-6zbq krb5kdc[25011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:42 dist-test-slave-6zbq krb5kdc[25011](info): set up 2 sockets
Aug 12 06:51:42 dist-test-slave-6zbq krb5kdc[25011](info): commencing operation
krb5kdc: starting...
W20260812 06:51:43.617684 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:43 dist-test-slave-6zbq krb5kdc[25011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:43.821679 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37081
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:44.130928 25035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:44.131176 25035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:44.131304 25035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:44.138121 25035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:44.138207 25035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:44.138273 25035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:44.138363 25035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:44.138432 25035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:44.147608 25035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:44.148693 25035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:44.149856 25035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:44.160391 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:44.160480 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:44.160800 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:44.161209 25035 server_base.cc:1034] running on GCE node
I20260812 06:51:44.161854 25035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:44.163075 25035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:44.164269 25035 hybrid_clock.cc:648] HybridClock initialized: now 1786517504164226 us; error 38 us; skew 500 ppm
I20260812 06:51:44.164573 25035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:44 dist-test-slave-6zbq krb5kdc[25011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:44.167680 25035 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:44.168944 25035 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:44.169059 25035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:44.169452 25035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:44.172331 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "3913f6cf1bdc418e9c116f8a09780495"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "4a6a6adf7c6db6b8a268db46be92d963"
server_key_iv: "33ee9c2671bed5f6ab7035e4a555eb41"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.173067 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "3913f6cf1bdc418e9c116f8a09780495"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "4a6a6adf7c6db6b8a268db46be92d963"
server_key_iv: "33ee9c2671bed5f6ab7035e4a555eb41"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.177661 25035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:51:44.180830 25060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:44.181991 25035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:44.182145 25035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "3913f6cf1bdc418e9c116f8a09780495"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "4a6a6adf7c6db6b8a268db46be92d963"
server_key_iv: "33ee9c2671bed5f6ab7035e4a555eb41"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.182374 25035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:44.209805 25035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:44.213193 25035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:44.213420 25035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:44.229923 25035 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:37081
I20260812 06:51:44.229943 25155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:37081 every 8 connection(s)
I20260812 06:51:44.230726 25035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:44.233944 25156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:44.238252 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25035
I20260812 06:51:44.238502 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:44.239279 17685 external_mini_cluster.cc:1428] Setting key 604040f556479c928842f16c94b8f349
I20260812 06:51:44.244711 25156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: Bootstrap starting.
I20260812 06:51:44.247948 25156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:44.249109 25156 log.cc:824] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:44.252315 25156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: No bootstrap required, opened a new log
Aug 12 06:51:44 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:44.260965 25156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } }
I20260812 06:51:44.261304 25156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:44.261404 25156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3913f6cf1bdc418e9c116f8a09780495, State: Initialized, Role: FOLLOWER
I20260812 06:51:44.261904 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:44.241284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.17.69.126:37081)
0812 06:51:44.242046 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:44.242057 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:44.242100 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:51:44.243839 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:44.243860 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:44.244200 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:44.244484 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:44.246320 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.247599 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.249274 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.250291 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.250487 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:44.255177 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:44.255184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:44.255200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:44.255237 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:44.258116 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:44.258885 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:44.258891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:44.258897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:44.259059 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:44.259433 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:44.259436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:44.259438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:44.259951 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:44.260162 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:44.260535 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 06:51:44.260850 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":557,"thread_start_us":349,"threads_started":1}
I20260812 06:51:44.262037 25156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } }
I20260812 06:51:44.262308 25156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:44.262475 25156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:44.262636 25156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:44.264789 25156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } }
I20260812 06:51:44.265233 25156 leader_election.cc:302] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3913f6cf1bdc418e9c116f8a09780495; no voters:
I20260812 06:51:44.265756 25156 leader_election.cc:288] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:44.265877 25163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:44.266537 25163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [term 1 LEADER]: Becoming Leader. State: Replica: 3913f6cf1bdc418e9c116f8a09780495, State: Running, Role: LEADER
I20260812 06:51:44.267091 25163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } }
I20260812 06:51:44.267694 25156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:44.269614 25167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3913f6cf1bdc418e9c116f8a09780495. Latest consensus state: current_term: 1 leader_uuid: "3913f6cf1bdc418e9c116f8a09780495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } } }
I20260812 06:51:44.269902 25167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:44.270076 25165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3913f6cf1bdc418e9c116f8a09780495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3913f6cf1bdc418e9c116f8a09780495" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37081 } } }
I20260812 06:51:44.270376 25165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:44.274745 25173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:44.278898 25173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:44.286193 25173 catalog_manager.cc:1347] Generated new cluster ID: 40a6ea2edd094783a14286cd234a2dc5
I20260812 06:51:44.286288 25173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:44.300346 25173 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:44.301488 25173 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:44.307801 25173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: Generated new TSK 0
I20260812 06:51:44.308504 25173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:44.370566 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:44.684583 25201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:44.684839 25201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:44.684984 25201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:44.691948 25201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:44.692051 25201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:44.692207 25201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:44.701563 25201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:44.702796 25201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:44.704063 25201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:44.716059 25210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:44.716065 25214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:44.716573 25207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:44.716866 25201 server_base.cc:1034] running on GCE node
I20260812 06:51:44.717442 25201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:44.718291 25201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:44.719497 25201 hybrid_clock.cc:648] HybridClock initialized: now 1786517504719445 us; error 49 us; skew 500 ppm
I20260812 06:51:44.719775 25201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:44 dist-test-slave-6zbq krb5kdc[25011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:44.722754 25201 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:44.723778 25201 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:44.723878 25201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:44.724166 25201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:44.727119 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "1c8e6658a5664865b67c23fbbd73e869"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "9a9ee9ddd5ae3b27f1df860c977d2509"
server_key_iv: "66ad692d86354a297d1b9822becc8f06"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.727905 25201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "1c8e6658a5664865b67c23fbbd73e869"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "9a9ee9ddd5ae3b27f1df860c977d2509"
server_key_iv: "66ad692d86354a297d1b9822becc8f06"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.732630 25201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:44.735812 25222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:44.736943 25201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:51:44.737090 25201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "1c8e6658a5664865b67c23fbbd73e869"
format_stamp: "Formatted at 2026-08-12 06:51:44 on dist-test-slave-6zbq"
server_key: "9a9ee9ddd5ae3b27f1df860c977d2509"
server_key_iv: "66ad692d86354a297d1b9822becc8f06"
server_key_version: "encryptionkey@0"
I20260812 06:51:44.737277 25201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:44.749547 25201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:44.753373 25201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:44.753610 25201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:44.754495 25201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:44.755744 25201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:44.755844 25201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:44.755945 25201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:44.756014 25201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:44.784838 25201 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:32783
I20260812 06:51:44.784904 25423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:32783 every 8 connection(s)
I20260812 06:51:44.785699 25201 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:44.787983 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25201
I20260812 06:51:44.788225 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:44.789038 17685 external_mini_cluster.cc:1428] Setting key b0b4c3f7ff84110ddbf5ac26bd570f23
Aug 12 06:51:44 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517504, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:44.805266 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:44.788135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:42865 (local address 127.17.69.126:37081)
0812 06:51:44.788623 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:44.788628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:44.789683 (+ 1055us) server_negotiation.cc:407] Connection header received
0812 06:51:44.791662 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:44.791667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:44.791747 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:44.791843 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:44.793292 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.794021 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.795545 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.795931 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.796038 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:44.799386 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:44.799391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:44.799395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:44.799427 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:44.802325 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:44.803038 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:44.803044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:44.803047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:44.803200 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:44.803579 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:44.803583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:44.803586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:44.803922 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:44.804090 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:44.804681 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 06:51:44.804891 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
I20260812 06:51:44.805977 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:44.788750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37081 (local address 127.17.69.65:42865)
0812 06:51:44.789530 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:44.789577 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:44.791420 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:44.792074 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:44.792081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:44.792570 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:44.793125 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:44.793141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.794174 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.794179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:44.795410 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:44.795422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:44.796143 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:44.796151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:44.796369 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:44.796956 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:44.796977 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:44.799168 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:44.802480 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:44.802486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:44.802497 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:44.802903 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:44.803294 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:44.803297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:44.803299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:44.803467 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:44.804037 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:44.804042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:44.804331 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 06:51:44.804538 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:51:44.804897 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":348,"threads_started":1}
I20260812 06:51:44.807545 25424 heartbeater.cc:347] Connected to a master server at 127.17.69.126:37081
I20260812 06:51:44.807813 25424 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:44.808508 25424 heartbeater.cc:511] Master 127.17.69.126:37081 requested a full tablet report, sending...
I20260812 06:51:44.810843 25101 ts_manager.cc:194] Registered new tserver with Master: 1c8e6658a5664865b67c23fbbd73e869 (127.17.69.65:32783)
I20260812 06:51:44.812834 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:42865
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:44.838150 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:45.154845 25437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:45.155092 25437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:45.155185 25437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:45.162542 25437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:45.162642 25437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:45.162758 25437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:45.173027 25437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:45.174188 25437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:45.175486 25437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:45.187592 25442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:45.187446 25443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:45.187558 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:45.188472 25437 server_base.cc:1034] running on GCE node
I20260812 06:51:45.189038 25437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:45.189854 25437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:45.191051 25437 hybrid_clock.cc:648] HybridClock initialized: now 1786517505191005 us; error 35 us; skew 500 ppm
I20260812 06:51:45.191339 25437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:45 dist-test-slave-6zbq krb5kdc[25011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517505, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:45.198092 25437 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:45.199168 25437 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:45.199268 25437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:45.199559 25437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:45.202307 25437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "9831eea47b5944cea2bbde23ce623b18"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "31c1ab4c9615aeef280da8e0b54a57b5"
server_key_iv: "8ffcc91dec54093d325a7ae452fcd285"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.203117 25437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "9831eea47b5944cea2bbde23ce623b18"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "31c1ab4c9615aeef280da8e0b54a57b5"
server_key_iv: "8ffcc91dec54093d325a7ae452fcd285"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.207713 25437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:45.210911 25460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.212126 25437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:51:45.212291 25437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "9831eea47b5944cea2bbde23ce623b18"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "31c1ab4c9615aeef280da8e0b54a57b5"
server_key_iv: "8ffcc91dec54093d325a7ae452fcd285"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.212488 25437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:45.245992 25437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:45.249606 25437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:45.249864 25437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:45.250795 25437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:45.252012 25437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:45.252095 25437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.252199 25437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:45.252266 25437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.280925 25437 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:40309
I20260812 06:51:45.280988 25660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:40309 every 8 connection(s)
I20260812 06:51:45.281690 25437 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:45.286394 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25437
I20260812 06:51:45.286594 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:45.287150 17685 external_mini_cluster.cc:1428] Setting key 1beb8166bc3f84c5022782ca9f607d9f
Aug 12 06:51:45 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517505, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:45.301951 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.283994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58417 (local address 127.17.69.126:37081)
0812 06:51:45.284172 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.284177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.285209 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 06:51:45.287443 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.287448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.287515 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.287598 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:45.289257 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.289983 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.292112 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.292513 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.292602 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.296331 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:45.296336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:45.296340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:45.296371 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:45.299083 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.299762 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.299766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.299769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.299904 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.300311 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.300314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.300317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.300575 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:45.300740 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.301401 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.301623 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:51:45.302855 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.284390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37081 (local address 127.17.69.66:58417)
0812 06:51:45.285050 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:45.285085 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:45.287138 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:45.287834 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:45.287842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:45.288412 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:45.289086 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.289105 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.290124 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.290129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.291983 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.291998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.292720 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.292726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.292970 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.293618 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:45.293645 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:45.296110 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:45.299214 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:45.299220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:45.299230 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:45.299620 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:45.300009 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:45.300013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:45.300015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:45.300193 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:45.300705 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:45.300712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:45.301016 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 06:51:45.301271 (+ 255us) client_negotiation.cc:240] Negotiation successful
0812 06:51:45.301757 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":272,"threads_started":1}
I20260812 06:51:45.304332 25662 heartbeater.cc:347] Connected to a master server at 127.17.69.126:37081
I20260812 06:51:45.304631 25662 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:45.305234 25662 heartbeater.cc:511] Master 127.17.69.126:37081 requested a full tablet report, sending...
I20260812 06:51:45.306721 25101 ts_manager.cc:194] Registered new tserver with Master: 9831eea47b5944cea2bbde23ce623b18 (127.17.69.66:40309)
I20260812 06:51:45.307498 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:58417
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:45.334730 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:45.646476 25667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:45.646710 25667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:45.646832 25667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:45.653776 25667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:45.653868 25667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:45.654007 25667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:45.663318 25667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:37081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:45.664516 25667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:45.665761 25667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:45.677516 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:45.677572 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:45.677601 25678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:45.678766 25667 server_base.cc:1034] running on GCE node
I20260812 06:51:45.679260 25667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:45.680070 25667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:45.681264 25667 hybrid_clock.cc:648] HybridClock initialized: now 1786517505681219 us; error 47 us; skew 500 ppm
I20260812 06:51:45.681543 25667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:45 dist-test-slave-6zbq krb5kdc[25011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517505, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:45.684424 25667 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:45.685484 25667 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:45.685572 25667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:45.685866 25667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:45.688568 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "70076bf4fbcb40c99106610846e10fbb"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "010de5feb74dc08ac35b22a18069a8e6"
server_key_iv: "297a3fa4b4551a706062ecfe84925051"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.689319 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "70076bf4fbcb40c99106610846e10fbb"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "010de5feb74dc08ac35b22a18069a8e6"
server_key_iv: "297a3fa4b4551a706062ecfe84925051"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.693876 25667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260812 06:51:45.697256 25684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.698447 25667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:51:45.698599 25667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "70076bf4fbcb40c99106610846e10fbb"
format_stamp: "Formatted at 2026-08-12 06:51:45 on dist-test-slave-6zbq"
server_key: "010de5feb74dc08ac35b22a18069a8e6"
server_key_iv: "297a3fa4b4551a706062ecfe84925051"
server_key_version: "encryptionkey@0"
I20260812 06:51:45.698786 25667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:45.717272 25667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:45.720875 25667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:45.721122 25667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:45.722002 25667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:45.723340 25667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:45.723428 25667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.723542 25667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:45.723620 25667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:45.753819 25667 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:46141
I20260812 06:51:45.753878 25899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:46141 every 8 connection(s)
I20260812 06:51:45.754681 25667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:45.763506 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25667
I20260812 06:51:45.763723 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:45.764250 17685 external_mini_cluster.cc:1428] Setting key 2b27cfd49d67eaa0e971088baa4382cc
Aug 12 06:51:45 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517505, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:45.772924 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.757023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:43423 (local address 127.17.69.126:37081)
0812 06:51:45.757218 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.757222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.758301 (+ 1079us) server_negotiation.cc:407] Connection header received
0812 06:51:45.760019 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.760023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.760090 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.760177 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:45.761569 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.762313 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.764127 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.764563 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.764658 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.768195 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:45.768200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:45.768203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:45.768233 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:45.770587 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.771213 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.771216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.771219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.771327 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.771631 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.771634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.771636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.771882 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:45.771987 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.772521 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.772683 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:51:45.773696 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.757397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37081 (local address 127.17.69.67:43423)
0812 06:51:45.758116 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:45.758184 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:45.759779 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:45.760354 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:45.760361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:45.760845 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:45.761414 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.761429 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.762458 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.762462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.763999 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.764014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.764767 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.764772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.764971 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.765645 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:45.765667 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:45.768025 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:45.770708 (+ 2683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:45.770714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:45.770725 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:45.771128 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:45.771399 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:45.771402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:45.771404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:45.771559 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:45.771972 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:45.771977 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:45.772229 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 06:51:45.772435 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 06:51:45.772761 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":266,"threads_started":1}
I20260812 06:51:45.775018 25900 heartbeater.cc:347] Connected to a master server at 127.17.69.126:37081
I20260812 06:51:45.775295 25900 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:45.775929 25900 heartbeater.cc:511] Master 127.17.69.126:37081 requested a full tablet report, sending...
I20260812 06:51:45.777259 25101 ts_manager.cc:194] Registered new tserver with Master: 70076bf4fbcb40c99106610846e10fbb (127.17.69.67:46141)
I20260812 06:51:45.778267 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:43423
I20260812 06:51:45.779330 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:51:45.795326 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.783642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.17.69.126:37081)
0812 06:51:45.783782 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.783786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.784276 (+ 490us) server_negotiation.cc:407] Connection header received
0812 06:51:45.784428 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.784433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.784496 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.784572 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:45.785991 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.786796 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.788028 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.788449 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.788554 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.790655 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:45.790660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:45.790663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:45.790691 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:45.792845 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.793440 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.793446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.793450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.793577 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:45.793923 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:45.793928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:45.793932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:45.794243 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:45.794488 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.794836 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.795039 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:51:45.799008 25101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:45.805711 25101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:45.815640 25424 heartbeater.cc:503] Master 127.17.69.126:37081 was elected leader, sending a full tablet report...
I20260812 06:51:45.828164 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.816539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:46141 (local address 127.0.0.1:40550)
0812 06:51:45.819193 (+ 2654us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:45.819231 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:45.819406 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:45.820285 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:45.820292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:45.820325 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:45.821575 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.821589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.823382 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.823387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.825797 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.825812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.827254 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.827260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.827465 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.827503 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:51:45.827604 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:51:45.827770 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2436,"mutex_wait_us":29,"thread_start_us":2302,"threads_started":1}
I20260812 06:51:45.828913 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.816663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.17.69.67:46141)
0812 06:51:45.818848 (+ 2185us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.818856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.819333 (+ 477us) server_negotiation.cc:407] Connection header received
0812 06:51:45.819584 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.819592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.819820 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.819998 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:45.821844 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.823256 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.825952 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.827074 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.827187 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.827406 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.827667 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.827822 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1991,"thread_start_us":1826,"threads_started":1}
I20260812 06:51:45.829195 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.816726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40309 (local address 127.0.0.1:40790)
0812 06:51:45.820932 (+ 4206us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:45.820948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:45.821065 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:45.822043 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:45.822047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:45.822069 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:45.822537 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.822551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.824226 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.824236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.826269 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.826280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.828306 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.828313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.828509 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.828614 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 06:51:45.828691 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:51:45.828821 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4092,"mutex_wait_us":86,"thread_start_us":3935,"threads_started":1}
I20260812 06:51:45.829246 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.820608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.17.69.65:32783)
0812 06:51:45.821319 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.821327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.821893 (+ 566us) server_negotiation.cc:407] Connection header received
0812 06:51:45.822034 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.822041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.822232 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.822414 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:45.823047 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.824195 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.826512 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.827832 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.827943 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.828146 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.828795 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.828944 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":355,"threads_started":1}
I20260812 06:51:45.829881 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.820105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:32783 (local address 127.0.0.1:38650)
0812 06:51:45.821781 (+ 1676us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:45.821798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:45.821907 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:45.822416 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:45.822420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:45.822449 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:45.822870 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.822883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.824401 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.824406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.826374 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:45.826389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.828319 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.828325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:45.828512 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.828615 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 06:51:45.828701 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 06:51:45.829531 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1549,"mutex_wait_us":833,"thread_start_us":1399,"threads_started":1}
I20260812 06:51:45.829958 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:45.817326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.17.69.66:40309)
0812 06:51:45.820304 (+ 2978us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:45.820313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:45.821502 (+ 1189us) server_negotiation.cc:407] Connection header received
0812 06:51:45.821602 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:45.821609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:45.821805 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:45.821969 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:45.822696 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.824069 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.826424 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:45.827799 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:45.827934 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:45.828120 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:45.829507 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0812 06:51:45.829663 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2801,"thread_start_us":2711,"threads_started":1}
I20260812 06:51:45.832590 25777 tablet_service.cc:1467] Processing CreateTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 (DEFAULT_TABLE table=test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:45.833290 25546 tablet_service.cc:1467] Processing CreateTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 (DEFAULT_TABLE table=test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:45.834456 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538508ed2e304cb2bcac9c43b8891d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:45.834514 25546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538508ed2e304cb2bcac9c43b8891d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:45.835191 25306 tablet_service.cc:1467] Processing CreateTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 (DEFAULT_TABLE table=test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:45.836206 25306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538508ed2e304cb2bcac9c43b8891d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:45.847163 25934 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Bootstrap starting.
I20260812 06:51:45.850332 25934 tablet_bootstrap.cc:654] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:45.851054 25936 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Bootstrap starting.
I20260812 06:51:45.851717 25934 log.cc:824] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:45.853096 25937 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Bootstrap starting.
I20260812 06:51:45.854460 25936 tablet_bootstrap.cc:654] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:45.854679 25934 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: No bootstrap required, opened a new log
I20260812 06:51:45.855005 25934 ts_tablet_manager.cc:1397] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260812 06:51:45.855665 25936 log.cc:824] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:45.856462 25937 tablet_bootstrap.cc:654] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:45.857549 25937 log.cc:824] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:45.858791 25936 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: No bootstrap required, opened a new log
I20260812 06:51:45.859037 25936 ts_tablet_manager.cc:1397] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260812 06:51:45.859865 25937 tablet_bootstrap.cc:492] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: No bootstrap required, opened a new log
I20260812 06:51:45.860080 25937 ts_tablet_manager.cc:1397] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 06:51:45.864449 25934 raft_consensus.cc:348] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.864849 25934 raft_consensus.cc:374] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:45.864944 25934 raft_consensus.cc:729] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8e6658a5664865b67c23fbbd73e869, State: Initialized, Role: FOLLOWER
I20260812 06:51:45.865528 25934 consensus_queue.cc:260] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.867179 25934 ts_tablet_manager.cc:1428] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260812 06:51:45.866964 25937 raft_consensus.cc:348] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.867328 25937 raft_consensus.cc:374] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:45.867434 25937 raft_consensus.cc:729] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70076bf4fbcb40c99106610846e10fbb, State: Initialized, Role: FOLLOWER
I20260812 06:51:45.868001 25937 consensus_queue.cc:260] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.868685 25936 raft_consensus.cc:348] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.869019 25936 raft_consensus.cc:374] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:45.869118 25936 raft_consensus.cc:729] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9831eea47b5944cea2bbde23ce623b18, State: Initialized, Role: FOLLOWER
I20260812 06:51:45.869606 25936 consensus_queue.cc:260] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:45.870579 25662 heartbeater.cc:503] Master 127.17.69.126:37081 was elected leader, sending a full tablet report...
I20260812 06:51:45.871189 25936 ts_tablet_manager.cc:1428] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 06:51:45.874908 25937 ts_tablet_manager.cc:1428] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260812 06:51:45.875077 25900 heartbeater.cc:503] Master 127.17.69.126:37081 was elected leader, sending a full tablet report...
W20260812 06:51:46.006069 25902 tablet.cc:2369] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:46.022710 25953 raft_consensus.cc:482] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:46.022984 25953 raft_consensus.cc:504] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:46.024575 25953 leader_election.cc:288] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9831eea47b5944cea2bbde23ce623b18 (127.17.69.66:40309), 1c8e6658a5664865b67c23fbbd73e869 (127.17.69.65:32783)
I20260812 06:51:46.031397 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.024991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:49035 (local address 127.17.69.66:40309)
0812 06:51:46.025203 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.025209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.025429 (+ 220us) server_negotiation.cc:407] Connection header received
0812 06:51:46.025609 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.025613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.025675 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.025769 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:46.026239 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.027255 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.029539 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.030634 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.030815 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.030888 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.031060 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.031182 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:51:46.031481 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.025020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40309 (local address 127.17.69.67:49035)
0812 06:51:46.025342 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:46.025357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:46.025481 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:46.025814 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:46.025818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:46.025848 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:46.026095 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:46.026104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.027384 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.027389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:46.029382 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:46.029401 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.030785 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.030789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:46.030916 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.030939 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:51:46.031012 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 06:51:46.031131 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":22,"spinlock_wait_cycles":20736}
I20260812 06:51:46.031909 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.025083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:32783 (local address 127.17.69.67:47459)
0812 06:51:46.025756 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:46.025773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:46.025890 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:46.026172 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:46.026176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:46.026201 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:51:46.026506 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:46.026515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.027758 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.027763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:46.029711 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:46.029722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.031183 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.031188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:46.031352 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.031382 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:51:46.031485 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:51:46.031599 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":25,"thread_start_us":395,"threads_started":1}
I20260812 06:51:46.032120 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.025095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:47459 (local address 127.17.69.65:32783)
0812 06:51:46.025297 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.025303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.025866 (+ 563us) server_negotiation.cc:407] Connection header received
0812 06:51:46.026002 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.026005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.026062 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.026126 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:51:46.026648 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.027615 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.029845 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.030987 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.031136 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.031211 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.031761 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.031900 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:51:46.032447 25586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538508ed2e304cb2bcac9c43b8891d64" candidate_uuid: "70076bf4fbcb40c99106610846e10fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9831eea47b5944cea2bbde23ce623b18" is_pre_election: true
I20260812 06:51:46.032989 25586 raft_consensus.cc:2454] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70076bf4fbcb40c99106610846e10fbb in term 0.
I20260812 06:51:46.033025 25346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538508ed2e304cb2bcac9c43b8891d64" candidate_uuid: "70076bf4fbcb40c99106610846e10fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8e6658a5664865b67c23fbbd73e869" is_pre_election: true
W20260812 06:51:46.033448 25663 tablet.cc:2369] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:46.033584 25346 raft_consensus.cc:2454] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70076bf4fbcb40c99106610846e10fbb in term 0.
I20260812 06:51:46.033759 25691 leader_election.cc:302] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70076bf4fbcb40c99106610846e10fbb, 9831eea47b5944cea2bbde23ce623b18; no voters:
I20260812 06:51:46.034188 25953 raft_consensus.cc:2781] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:46.034322 25953 raft_consensus.cc:482] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:46.034446 25953 raft_consensus.cc:3037] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:46.036515 25953 raft_consensus.cc:504] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:46.037061 25953 leader_election.cc:288] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [CANDIDATE]: Term 1 election: Requested vote from peers 9831eea47b5944cea2bbde23ce623b18 (127.17.69.66:40309), 1c8e6658a5664865b67c23fbbd73e869 (127.17.69.65:32783)
I20260812 06:51:46.037421 25586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538508ed2e304cb2bcac9c43b8891d64" candidate_uuid: "70076bf4fbcb40c99106610846e10fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9831eea47b5944cea2bbde23ce623b18"
I20260812 06:51:46.037436 25346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538508ed2e304cb2bcac9c43b8891d64" candidate_uuid: "70076bf4fbcb40c99106610846e10fbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c8e6658a5664865b67c23fbbd73e869"
I20260812 06:51:46.037585 25586 raft_consensus.cc:3037] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:46.037600 25346 raft_consensus.cc:3037] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 0 FOLLOWER]: Advancing to term 1
W20260812 06:51:46.037791 25430 tablet.cc:2369] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:46.039741 25586 raft_consensus.cc:2454] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70076bf4fbcb40c99106610846e10fbb in term 1.
I20260812 06:51:46.040040 25346 raft_consensus.cc:2454] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70076bf4fbcb40c99106610846e10fbb in term 1.
I20260812 06:51:46.040094 25691 leader_election.cc:302] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70076bf4fbcb40c99106610846e10fbb, 9831eea47b5944cea2bbde23ce623b18; no voters:
I20260812 06:51:46.040385 25953 raft_consensus.cc:2781] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:46.040853 25953 raft_consensus.cc:686] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 1 LEADER]: Becoming Leader. State: Replica: 70076bf4fbcb40c99106610846e10fbb, State: Running, Role: LEADER
I20260812 06:51:46.041388 25953 consensus_queue.cc:237] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } }
I20260812 06:51:46.045807 25101 catalog_manager.cc:5495] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 70076bf4fbcb40c99106610846e10fbb (127.17.69.67). New cstate: current_term: 1 leader_uuid: "70076bf4fbcb40c99106610846e10fbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70076bf4fbcb40c99106610846e10fbb" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 46141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 } health_report { overall_health: UNKNOWN } } }
I20260812 06:51:46.062875 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.057831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.17.69.67:46141)
0812 06:51:46.058008 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.058014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.058148 (+ 134us) server_negotiation.cc:407] Connection header received
0812 06:51:46.058372 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.058378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.058459 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.058577 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:51:46.058933 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.059630 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.060909 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.061301 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.061405 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.061855 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:51:46.062307 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:51:46.062468 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.062559 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.062636 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 06:51:46.070259 25346 raft_consensus.cc:1261] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 1 FOLLOWER]: Refusing update from remote peer 70076bf4fbcb40c99106610846e10fbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:46.070259 25586 raft_consensus.cc:1261] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 1 FOLLOWER]: Refusing update from remote peer 70076bf4fbcb40c99106610846e10fbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:46.071205 25956 consensus_queue.cc:1035] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c8e6658a5664865b67c23fbbd73e869" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:46.071594 25953 consensus_queue.cc:1035] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9831eea47b5944cea2bbde23ce623b18" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:46.082227 25963 mvcc.cc:205] Tried to move back new op lower bound from 7317575704854282240 to 7317575704748912640. Current Snapshot: MvccSnapshot[applied={T|T < 7317575704854282240}]
I20260812 06:51:46.096498 25101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:46.096719 25101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:46.100069 25101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3913f6cf1bdc418e9c116f8a09780495: Sending DeleteTablet for 3 replicas of tablet 538508ed2e304cb2bcac9c43b8891d64
I20260812 06:51:46.101117 25777 tablet_service.cc:1514] Processing DeleteTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:46 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:40550
I20260812 06:51:46.101117 25546 tablet_service.cc:1514] Processing DeleteTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:46 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:40790
I20260812 06:51:46.101372 25306 tablet_service.cc:1514] Processing DeleteTablet for tablet 538508ed2e304cb2bcac9c43b8891d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:46 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:38650
I20260812 06:51:46.107888 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25201
I20260812 06:51:46.114264 25977 tablet_replica.cc:330] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: stopping tablet replica
I20260812 06:51:46.114701 25977 raft_consensus.cc:2229] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:46.114986 25975 tablet_replica.cc:330] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: stopping tablet replica
I20260812 06:51:46.115336 25977 raft_consensus.cc:2258] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:46.115489 25975 raft_consensus.cc:2229] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:46.116034 25975 raft_consensus.cc:2258] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:46.119455 25977 ts_tablet_manager.cc:1905] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:46.118274 25975 ts_tablet_manager.cc:1905] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:46.126667 25977 ts_tablet_manager.cc:1918] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:46.126825 25977 log.cc:1196] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/538508ed2e304cb2bcac9c43b8891d64
I20260812 06:51:46.127344 25977 ts_tablet_manager.cc:1939] T 538508ed2e304cb2bcac9c43b8891d64 P 70076bf4fbcb40c99106610846e10fbb: Deleting consensus metadata
I20260812 06:51:46.127465 25978 tablet_replica.cc:330] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: stopping tablet replica
I20260812 06:51:46.127862 25978 raft_consensus.cc:2229] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:46.128328 25978 raft_consensus.cc:2258] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:46.129019 25061 catalog_manager.cc:4841] TS 70076bf4fbcb40c99106610846e10fbb (127.17.69.67:46141): tablet 538508ed2e304cb2bcac9c43b8891d64 (table test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]) successfully deleted
Aug 12 06:51:46 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:46.132154 25978 ts_tablet_manager.cc:1905] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:46.135030 25975 ts_tablet_manager.cc:1918] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:46.135195 25975 log.cc:1196] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/538508ed2e304cb2bcac9c43b8891d64
I20260812 06:51:46.135702 25975 ts_tablet_manager.cc:1939] T 538508ed2e304cb2bcac9c43b8891d64 P 9831eea47b5944cea2bbde23ce623b18: Deleting consensus metadata
I20260812 06:51:46.137387 25070 catalog_manager.cc:4841] TS 9831eea47b5944cea2bbde23ce623b18 (127.17.69.66:40309): tablet 538508ed2e304cb2bcac9c43b8891d64 (table test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]) successfully deleted
I20260812 06:51:46.139442 25978 ts_tablet_manager.cc:1918] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:46.139559 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.109262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.17.69.65:32783)
0812 06:51:46.109456 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.109463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.119679 (+ 10216us) server_negotiation.cc:407] Connection header received
0812 06:51:46.120091 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.120097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.120178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.120251 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:46.121930 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.123096 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.124881 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.125534 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.125626 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.132629 (+ 7003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:46.132643 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:46.132664 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:46.132714 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:46.136567 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.137403 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.137412 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.137415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.137606 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.138006 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.138011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.138015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.138419 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:46.138719 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.138974 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.139186 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:51:46.139636 25978 log.cc:1196] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/538508ed2e304cb2bcac9c43b8891d64
I20260812 06:51:46.140144 25978 ts_tablet_manager.cc:1939] T 538508ed2e304cb2bcac9c43b8891d64 P 1c8e6658a5664865b67c23fbbd73e869: Deleting consensus metadata
I20260812 06:51:46.140264 25418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:46.229187 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25201
I20260812 06:51:46.229467 25072 catalog_manager.cc:4841] TS 1c8e6658a5664865b67c23fbbd73e869 (127.17.69.65:32783): tablet 538508ed2e304cb2bcac9c43b8891d64 (table test-table [id=b796ac24e7f2467f8a9ffbc5efc5b839]) successfully deleted
I20260812 06:51:46.253402 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25437
Aug 12 06:51:46 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:46.268512 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.254200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.17.69.66:40309)
0812 06:51:46.254454 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.254460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.254501 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:51:46.254708 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.254714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.254800 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.254883 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:46.256094 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.256852 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.258177 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.258640 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.258752 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.262773 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:46.262788 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:46.262803 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:46.262835 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:46.265467 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.266093 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.266101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.266104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.266249 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.266879 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.266884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.266887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.267278 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:46.267559 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.267908 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.268190 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260812 06:51:46.269208 25655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:46.356487 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25437
I20260812 06:51:46.380535 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25667
Aug 12 06:51:46 dist-test-slave-6zbq krb5kdc[25011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:46.397359 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:46.381345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.17.69.67:46141)
0812 06:51:46.381527 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:46.381533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:46.381555 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:51:46.381780 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:46.381784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:46.381852 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:46.381935 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:46.383250 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.384082 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.385322 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:46.385811 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:46.385954 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:46.390308 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:46.390328 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:46.390379 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:46.390427 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:46.394502 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.395130 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.395140 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.395143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.395372 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:46.395743 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:46.395748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:46.395751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:46.396228 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:46.396561 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:46.396719 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 06:51:46.396930 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:51:46.398108 25894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:46.515771 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25667
I20260812 06:51:46.539222 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25035
I20260812 06:51:46.540278 25145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:46.648689 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25035
2026-08-12T06:51:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4268 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T06:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:46Z Disabled control of system clock
I20260812 06:51:46.723721 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38705
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:51:47.035077 25991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:47.035360 25991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:47.035444 25991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:47.042743 25991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:47.042855 25991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:47.042892 25991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:47.042948 25991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:47.042991 25991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:47.052594 25991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:47.053777 25991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:47.055510 25991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:47.067058 25998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.067058 25996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.067116 26001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:47.067975 25991 server_base.cc:1034] running on GCE node
I20260812 06:51:47.068681 25991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:47.069963 25991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:47.071154 25991 hybrid_clock.cc:648] HybridClock initialized: now 1786517507071109 us; error 41 us; skew 500 ppm
I20260812 06:51:47.071472 25991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:47.073500 25991 webserver.cc:460] Webserver started at http://127.17.69.126:36665/ using document root <none> and password file <none>
I20260812 06:51:47.074128 25991 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:47.074231 25991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:47.074574 25991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:47.077528 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "b473490534664d1880b307b576f0352f"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "80473b2ff8fe47d59b06653aefde697a"
server_key_iv: "f9331b026340c46e37aca45350708422"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.078282 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "b473490534664d1880b307b576f0352f"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "80473b2ff8fe47d59b06653aefde697a"
server_key_iv: "f9331b026340c46e37aca45350708422"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.083149 25991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:51:47.086200 26014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:47.087423 25991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:47.087572 25991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "b473490534664d1880b307b576f0352f"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "80473b2ff8fe47d59b06653aefde697a"
server_key_iv: "f9331b026340c46e37aca45350708422"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.087767 25991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:47.093259 25991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:47.093947 25991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:47.094213 25991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:47.111441 25991 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38705
I20260812 06:51:47.111506 26099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38705 every 8 connection(s)
I20260812 06:51:47.112718 25991 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:47.116201 26101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:47.120651 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25991
I20260812 06:51:47.120839 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:47.121717 17685 external_mini_cluster.cc:1428] Setting key aa6d1105d2d46dffb12c4f10c5f44350
I20260812 06:51:47.126889 26101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: Bootstrap starting.
I20260812 06:51:47.129772 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:47.123818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.17.69.126:38705)
0812 06:51:47.124826 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:47.124839 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:47.124882 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:51:47.126480 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:47.126502 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:47.126881 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:47.127226 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:47.128270 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:47.128277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:47.128288 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:47.128303 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:47.128365 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:47.128384 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:47.128584 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:47.128728 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 06:51:47.128949 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":363,"threads_started":1}
I20260812 06:51:47.129889 26101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:47.130955 26101 log.cc:824] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:47.133559 26101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: No bootstrap required, opened a new log
I20260812 06:51:47.140501 26101 raft_consensus.cc:348] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } }
I20260812 06:51:47.140808 26101 raft_consensus.cc:374] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:47.140900 26101 raft_consensus.cc:729] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b473490534664d1880b307b576f0352f, State: Initialized, Role: FOLLOWER
I20260812 06:51:47.141400 26101 consensus_queue.cc:260] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } }
I20260812 06:51:47.141604 26101 raft_consensus.cc:388] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:47.141734 26101 raft_consensus.cc:482] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:47.141878 26101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:47.144104 26101 raft_consensus.cc:504] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } }
I20260812 06:51:47.144533 26101 leader_election.cc:302] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b473490534664d1880b307b576f0352f; no voters:
I20260812 06:51:47.145063 26101 leader_election.cc:288] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:47.145198 26109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:47.145922 26109 raft_consensus.cc:686] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [term 1 LEADER]: Becoming Leader. State: Replica: b473490534664d1880b307b576f0352f, State: Running, Role: LEADER
I20260812 06:51:47.146479 26109 consensus_queue.cc:237] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } }
I20260812 06:51:47.147020 26101 sys_catalog.cc:564] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:47.149256 26118 sys_catalog.cc:455] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b473490534664d1880b307b576f0352f. Latest consensus state: current_term: 1 leader_uuid: "b473490534664d1880b307b576f0352f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } } }
I20260812 06:51:47.149655 26118 sys_catalog.cc:458] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:47.149722 26113 sys_catalog.cc:455] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b473490534664d1880b307b576f0352f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b473490534664d1880b307b576f0352f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38705 } } }
I20260812 06:51:47.150063 26113 sys_catalog.cc:458] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:47.151144 26129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:47.155808 26129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:47.164503 26129 catalog_manager.cc:1347] Generated new cluster ID: fa2dedd40bc6469b8871a9b3d8344fa0
I20260812 06:51:47.164594 26129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:47.189070 26129 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:47.189420 26129 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:47.195931 26129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: Generated new TSK 0
I20260812 06:51:47.196624 26129 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:51:47.208534 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--builtin_ntp_servers=127.17.69.84:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 06:51:47.524338 26143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:47.524587 26143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:47.524711 26143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:47.531872 26143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:47.531961 26143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:47.532100 26143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:47.540807 26143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:47.541991 26143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:47.543740 26143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:47.555992 26151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.555966 26155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.556180 26164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:47.557315 26143 server_base.cc:1034] running on GCE node
I20260812 06:51:47.557816 26143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:47.558746 26143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:47.559949 26143 hybrid_clock.cc:648] HybridClock initialized: now 1786517507559894 us; error 47 us; skew 500 ppm
I20260812 06:51:47.560246 26143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:47.562389 26143 webserver.cc:460] Webserver started at http://127.17.69.65:44583/ using document root <none> and password file <none>
I20260812 06:51:47.563011 26143 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:47.563103 26143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:47.563418 26143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:47.566064 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "a2a9c3a506cf41efae45b8f60e223b7d"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "6e290b62831bd5840e33b21468533691"
server_key_iv: "b9bfd21014ceddaf08ad5e38379fb1a9"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.566939 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "a2a9c3a506cf41efae45b8f60e223b7d"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "6e290b62831bd5840e33b21468533691"
server_key_iv: "b9bfd21014ceddaf08ad5e38379fb1a9"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.571903 26143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:51:47.575078 26179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:47.576239 26143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:47.576378 26143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "a2a9c3a506cf41efae45b8f60e223b7d"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "6e290b62831bd5840e33b21468533691"
server_key_iv: "b9bfd21014ceddaf08ad5e38379fb1a9"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.576571 26143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:47.581936 26143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:47.582664 26143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:47.582911 26143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:47.583745 26143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:47.585045 26143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:47.585139 26143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:47.585243 26143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:47.585311 26143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:47.615814 26143 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:38495
I20260812 06:51:47.615863 26389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:38495 every 8 connection(s)
I20260812 06:51:47.617059 26143 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:47.619545 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26143
I20260812 06:51:47.619761 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:47.620599 17685 external_mini_cluster.cc:1428] Setting key 44032148a931ffae2419983e42791cbb
I20260812 06:51:47.623576 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--builtin_ntp_servers=127.17.69.84:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:51:47.625684 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:47.619649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:48213 (local address 127.17.69.126:38705)
0812 06:51:47.619823 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:47.619829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:47.621711 (+ 1882us) server_negotiation.cc:407] Connection header received
0812 06:51:47.623484 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:47.623489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:47.623553 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:47.623672 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:47.624939 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:47.624945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:47.624950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:47.624972 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:47.625011 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:47.625026 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:47.625154 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:47.625372 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 06:51:47.625436 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:51:47.626515 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:47.620307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38705 (local address 127.17.69.65:48213)
0812 06:51:47.621321 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:47.621368 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:47.623226 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:47.623905 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:47.623912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:47.623998 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:47.624639 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:47.624641 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:47.624677 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:47.624762 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:47.625172 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:47.625183 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:47.625190 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:51:47.625416 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 06:51:47.625601 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":336,"threads_started":1}
I20260812 06:51:47.627815 26390 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38705
I20260812 06:51:47.628072 26390 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:47.628605 26390 heartbeater.cc:511] Master 127.17.69.126:38705 requested a full tablet report, sending...
I20260812 06:51:47.630468 26044 ts_manager.cc:194] Registered new tserver with Master: a2a9c3a506cf41efae45b8f60e223b7d (127.17.69.65:38495)
W20260812 06:51:47.943743 26395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:47.943988 26395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:47.944082 26395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:47.950969 26395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:47.951051 26395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:47.951189 26395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:47.959729 26395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:47.960870 26395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:47.962540 26395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:47.974831 26404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.974779 26407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:47.974854 26410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:47.975594 26395 server_base.cc:1034] running on GCE node
I20260812 06:51:47.976145 26395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:47.977007 26395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:47.978200 26395 hybrid_clock.cc:648] HybridClock initialized: now 1786517507978160 us; error 29 us; skew 500 ppm
I20260812 06:51:47.978504 26395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:47.980705 26395 webserver.cc:460] Webserver started at http://127.17.69.66:45877/ using document root <none> and password file <none>
I20260812 06:51:47.981369 26395 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:47.981462 26395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:47.981775 26395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:47.984524 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "1386c7fe87eb4b8882dd3427aa6cc643"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "e1b1257926fb9ec01c823d056e2aa25f"
server_key_iv: "74d526c6d5159f0285a2e622210d7162"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.985268 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "1386c7fe87eb4b8882dd3427aa6cc643"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "e1b1257926fb9ec01c823d056e2aa25f"
server_key_iv: "74d526c6d5159f0285a2e622210d7162"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.989877 26395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:51:47.993207 26421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:47.994457 26395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:51:47.994598 26395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "1386c7fe87eb4b8882dd3427aa6cc643"
format_stamp: "Formatted at 2026-08-12 06:51:47 on dist-test-slave-6zbq"
server_key: "e1b1257926fb9ec01c823d056e2aa25f"
server_key_iv: "74d526c6d5159f0285a2e622210d7162"
server_key_version: "encryptionkey@0"
I20260812 06:51:47.994812 26395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:47.999871 26395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:48.000572 26395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:48.000823 26395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:48.001645 26395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:48.002945 26395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:48.003036 26395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:48.003137 26395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:48.003208 26395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:48.031754 26395 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:36823
I20260812 06:51:48.031816 26613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:36823 every 8 connection(s)
I20260812 06:51:48.032995 26395 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:48.034945 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26395
I20260812 06:51:48.035094 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:48.035616 17685 external_mini_cluster.cc:1428] Setting key cb9b0f530cd1b4ea36a8172f44008875
I20260812 06:51:48.038635 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--builtin_ntp_servers=127.17.69.84:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 06:51:48.041358 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.035442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:35231 (local address 127.17.69.126:38705)
0812 06:51:48.035663 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.035670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.037051 (+ 1381us) server_negotiation.cc:407] Connection header received
0812 06:51:48.039276 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.039281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.039331 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.039409 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.040756 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.040761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.040764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.040779 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.040816 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.040827 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.040920 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.041095 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.041140 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:51:48.042501 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.035958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38705 (local address 127.17.69.66:35231)
0812 06:51:48.036910 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.036945 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.039001 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.039653 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.039664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.039782 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.040445 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.040448 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.040493 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.040591 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.040905 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.040914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.040921 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.041194 (+ 273us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.041426 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":298,"threads_started":1}
I20260812 06:51:48.043689 26614 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38705
I20260812 06:51:48.043967 26614 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:48.044497 26614 heartbeater.cc:511] Master 127.17.69.126:38705 requested a full tablet report, sending...
I20260812 06:51:48.045720 26044 ts_manager.cc:194] Registered new tserver with Master: 1386c7fe87eb4b8882dd3427aa6cc643 (127.17.69.66:36823)
W20260812 06:51:48.354192 26618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:48.354481 26618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:48.354624 26618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:48.361634 26618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:48.361716 26618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:48.361847 26618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:48.370545 26618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:48.371637 26618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:48.373245 26618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:48.385352 26634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:48.385406 26629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:48.385532 26628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:48.386555 26618 server_base.cc:1034] running on GCE node
I20260812 06:51:48.387081 26618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:48.387895 26618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:48.389081 26618 hybrid_clock.cc:648] HybridClock initialized: now 1786517508389031 us; error 43 us; skew 500 ppm
I20260812 06:51:48.389374 26618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:48.391403 26618 webserver.cc:460] Webserver started at http://127.17.69.67:45191/ using document root <none> and password file <none>
I20260812 06:51:48.391999 26618 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:48.392096 26618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:48.392396 26618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:48.395035 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "eab0e9e8612f4483b27cb8566cf379e4"
format_stamp: "Formatted at 2026-08-12 06:51:48 on dist-test-slave-6zbq"
server_key: "790be3c5e707986b52061e536611e073"
server_key_iv: "8a1a9564a76dc84315470b7874176471"
server_key_version: "encryptionkey@0"
I20260812 06:51:48.395777 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "eab0e9e8612f4483b27cb8566cf379e4"
format_stamp: "Formatted at 2026-08-12 06:51:48 on dist-test-slave-6zbq"
server_key: "790be3c5e707986b52061e536611e073"
server_key_iv: "8a1a9564a76dc84315470b7874176471"
server_key_version: "encryptionkey@0"
I20260812 06:51:48.400368 26618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:51:48.403431 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:48.404578 26618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:48.404719 26618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "eab0e9e8612f4483b27cb8566cf379e4"
format_stamp: "Formatted at 2026-08-12 06:51:48 on dist-test-slave-6zbq"
server_key: "790be3c5e707986b52061e536611e073"
server_key_iv: "8a1a9564a76dc84315470b7874176471"
server_key_version: "encryptionkey@0"
I20260812 06:51:48.404904 26618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:48.409958 26618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:48.410693 26618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:48.410920 26618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:48.411741 26618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:48.412986 26618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:48.413077 26618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:48.413180 26618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:48.413249 26618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:48.442292 26618 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:43803
I20260812 06:51:48.442333 26844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:43803 every 8 connection(s)
I20260812 06:51:48.443456 26618 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:48.450525 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26618
I20260812 06:51:48.450703 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:48.450906 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.445671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:55205 (local address 127.17.69.126:38705)
0812 06:51:48.445906 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.445913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.447128 (+ 1215us) server_negotiation.cc:407] Connection header received
0812 06:51:48.448828 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.448832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.448887 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.448967 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.450120 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.450127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.450132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.450162 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.450204 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.450221 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.450378 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.450568 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.450637 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:51:48.451246 17685 external_mini_cluster.cc:1428] Setting key 5321c9efcd2db241782c34794c3bca59
I20260812 06:51:48.451715 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.446110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38705 (local address 127.17.69.67:55205)
0812 06:51:48.446991 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.447026 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.448587 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.449164 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.449172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.449253 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.449832 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.449834 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.449868 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.449943 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.450330 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.450374 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.450383 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.450601 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.450829 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":273,"threads_started":1}
I20260812 06:51:48.452941 26845 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38705
I20260812 06:51:48.453195 26845 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:48.453733 26845 heartbeater.cc:511] Master 127.17.69.126:38705 requested a full tablet report, sending...
I20260812 06:51:48.455019 26044 ts_manager.cc:194] Registered new tserver with Master: eab0e9e8612f4483b27cb8566cf379e4 (127.17.69.67:43803)
I20260812 06:51:48.465806 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:51:48.472198 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.469914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.17.69.126:38705)
0812 06:51:48.470047 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.470052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.470553 (+ 501us) server_negotiation.cc:407] Connection header received
0812 06:51:48.470713 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.470716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.470762 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.470838 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.471618 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.471624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.471629 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.471646 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.471676 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.471687 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.471792 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.471888 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.471944 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:51:48.475442 26044 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:51:48.481827 26044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:51:48.499539 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.495104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55040)
0812 06:51:48.495555 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.495572 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.495668 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.497603 (+ 1935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.497610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.497691 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.498412 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.498416 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.498442 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.498504 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.498910 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.498922 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.498929 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.499081 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.499168 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":746,"threads_started":1}
I20260812 06:51:48.500391 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.495641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.17.69.65:38495)
0812 06:51:48.497152 (+ 1511us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.497160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.497183 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:51:48.497236 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.497242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.497387 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.497577 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.498724 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.498733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.498744 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.498764 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.498807 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.498822 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.498933 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.499070 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.499165 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":1262,"threads_started":1}
I20260812 06:51:48.503288 26264 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c (DEFAULT_TABLE table=test-table [id=55078b04285448d8a0de231016b51c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:48.504633 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13564ea8e44e27a53da3696b2daf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:48.507153 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.493004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36823 (local address 127.0.0.1:58606)
0812 06:51:48.494988 (+ 1984us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.495003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.495087 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.504917 (+ 9830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.504922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.504944 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.505590 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.505593 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.505610 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.505652 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.506590 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.506595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.506600 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.506728 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.506779 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1855,"thread_start_us":930,"threads_started":1}
I20260812 06:51:48.507411 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.493631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.17.69.66:36823)
0812 06:51:48.504427 (+ 10796us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.504435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.504466 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:51:48.504535 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.504543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.504739 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.504899 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.506364 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.506374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.506386 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.506410 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.506462 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.506481 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.506650 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.506715 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.506884 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10605,"thread_start_us":11860,"threads_started":1}
I20260812 06:51:48.510489 26506 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c (DEFAULT_TABLE table=test-table [id=55078b04285448d8a0de231016b51c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:48.511930 26506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13564ea8e44e27a53da3696b2daf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:48.512820 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.492502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:43803 (local address 127.0.0.1:46656)
0812 06:51:48.494332 (+ 1830us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.494393 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.494522 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.511081 (+ 16559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.511086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.511109 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.511678 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.511681 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.511697 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.511740 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.512270 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.512274 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.512279 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.512380 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.512429 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1579,"thread_start_us":2231,"threads_started":1}
I20260812 06:51:48.512838 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.492744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.17.69.67:43803)
0812 06:51:48.508021 (+ 15277us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.508033 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.508056 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:51:48.508105 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.508109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.508251 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.511095 (+ 2844us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.512052 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.512060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.512072 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.512088 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.512136 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.512154 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.512318 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.512383 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.512464 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15144,"thread_start_us":14980,"threads_started":1}
I20260812 06:51:48.516232 26728 tablet_service.cc:1467] Processing CreateTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c (DEFAULT_TABLE table=test-table [id=55078b04285448d8a0de231016b51c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:51:48.516970 26892 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Bootstrap starting.
I20260812 06:51:48.518134 26728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e13564ea8e44e27a53da3696b2daf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:48.519853 26892 tablet_bootstrap.cc:654] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:48.520776 26892 log.cc:824] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:48.523110 26892 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: No bootstrap required, opened a new log
I20260812 06:51:48.523342 26892 ts_tablet_manager.cc:1397] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260812 06:51:48.525323 26894 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Bootstrap starting.
I20260812 06:51:48.528653 26894 tablet_bootstrap.cc:654] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:48.529788 26894 log.cc:824] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:48.530124 26895 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Bootstrap starting.
I20260812 06:51:48.532644 26892 raft_consensus.cc:348] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.533078 26892 raft_consensus.cc:374] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:48.533215 26895 tablet_bootstrap.cc:654] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:48.533195 26892 raft_consensus.cc:729] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a9c3a506cf41efae45b8f60e223b7d, State: Initialized, Role: FOLLOWER
I20260812 06:51:48.533576 26894 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: No bootstrap required, opened a new log
I20260812 06:51:48.533828 26894 ts_tablet_manager.cc:1397] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260812 06:51:48.533779 26892 consensus_queue.cc:260] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.534251 26895 log.cc:824] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:48.534871 26390 heartbeater.cc:503] Master 127.17.69.126:38705 was elected leader, sending a full tablet report...
I20260812 06:51:48.535447 26892 ts_tablet_manager.cc:1428] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260812 06:51:48.536799 26895 tablet_bootstrap.cc:492] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: No bootstrap required, opened a new log
I20260812 06:51:48.537020 26895 ts_tablet_manager.cc:1397] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:51:48.543200 26894 raft_consensus.cc:348] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.543646 26894 raft_consensus.cc:374] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:48.543759 26894 raft_consensus.cc:729] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1386c7fe87eb4b8882dd3427aa6cc643, State: Initialized, Role: FOLLOWER
I20260812 06:51:48.544241 26894 consensus_queue.cc:260] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.544315 26895 raft_consensus.cc:348] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.544641 26895 raft_consensus.cc:374] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:48.544735 26895 raft_consensus.cc:729] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab0e9e8612f4483b27cb8566cf379e4, State: Initialized, Role: FOLLOWER
I20260812 06:51:48.545107 26614 heartbeater.cc:503] Master 127.17.69.126:38705 was elected leader, sending a full tablet report...
I20260812 06:51:48.545213 26895 consensus_queue.cc:260] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.545715 26894 ts_tablet_manager.cc:1428] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Time spent starting tablet: real 0.012s user 0.003s sys 0.008s
I20260812 06:51:48.546150 26845 heartbeater.cc:503] Master 127.17.69.126:38705 was elected leader, sending a full tablet report...
I20260812 06:51:48.546733 26895 ts_tablet_manager.cc:1428] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260812 06:51:48.617794 26904 raft_consensus.cc:482] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:51:48.618058 26904 raft_consensus.cc:504] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
W20260812 06:51:48.619294 26391 tablet.cc:2369] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:51:48.621042 26904 leader_election.cc:288] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a9c3a506cf41efae45b8f60e223b7d (127.17.69.65:38495), 1386c7fe87eb4b8882dd3427aa6cc643 (127.17.69.66:36823)
I20260812 06:51:48.622771 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.620815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:43859 (local address 127.17.69.66:36823)
0812 06:51:48.620990 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.620995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.621012 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:51:48.621065 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.621085 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.621141 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.621233 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.622111 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.622116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.622121 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.622138 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.622173 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.622188 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.622298 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.622494 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.622552 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:51:48.622970 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.619963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36823 (local address 127.17.69.67:43859)
0812 06:51:48.620632 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.620648 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.620743 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.621332 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.621337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.621360 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.621904 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.621908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.621924 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.621966 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.622287 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.622291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.622296 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.622496 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.622544 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":843,"threads_started":1}
I20260812 06:51:48.622982 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.620154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:35307 (local address 127.17.69.65:38495)
0812 06:51:48.620362 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.620369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.620541 (+ 172us) server_negotiation.cc:407] Connection header received
0812 06:51:48.620694 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.620698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.620758 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.620847 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.621933 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.621938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.621941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.621956 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.621985 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.621996 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.622092 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.622674 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.622732 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:51:48.622581 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.619927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.17.69.67:35307)
0812 06:51:48.620460 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.620478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:48.620576 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:48.621016 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:48.621020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:48.621049 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:48.621701 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:51:48.621705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:48.621723 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:51:48.621774 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:48.622143 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:48.622148 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:48.622152 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 06:51:48.622252 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 06:51:48.622317 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418}
I20260812 06:51:48.623907 26533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13564ea8e44e27a53da3696b2daf1c" candidate_uuid: "eab0e9e8612f4483b27cb8566cf379e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" is_pre_election: true
I20260812 06:51:48.624154 26307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13564ea8e44e27a53da3696b2daf1c" candidate_uuid: "eab0e9e8612f4483b27cb8566cf379e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" is_pre_election: true
I20260812 06:51:48.624343 26533 raft_consensus.cc:2454] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab0e9e8612f4483b27cb8566cf379e4 in term 0.
I20260812 06:51:48.624647 26307 raft_consensus.cc:2454] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab0e9e8612f4483b27cb8566cf379e4 in term 0.
I20260812 06:51:48.624946 26652 leader_election.cc:302] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1386c7fe87eb4b8882dd3427aa6cc643, eab0e9e8612f4483b27cb8566cf379e4; no voters:
I20260812 06:51:48.625325 26904 raft_consensus.cc:2781] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:51:48.625437 26904 raft_consensus.cc:482] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:51:48.625505 26904 raft_consensus.cc:3037] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:48.627664 26904 raft_consensus.cc:504] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.628213 26904 leader_election.cc:288] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [CANDIDATE]: Term 1 election: Requested vote from peers a2a9c3a506cf41efae45b8f60e223b7d (127.17.69.65:38495), 1386c7fe87eb4b8882dd3427aa6cc643 (127.17.69.66:36823)
I20260812 06:51:48.628598 26307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13564ea8e44e27a53da3696b2daf1c" candidate_uuid: "eab0e9e8612f4483b27cb8566cf379e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a9c3a506cf41efae45b8f60e223b7d"
I20260812 06:51:48.628681 26533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e13564ea8e44e27a53da3696b2daf1c" candidate_uuid: "eab0e9e8612f4483b27cb8566cf379e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1386c7fe87eb4b8882dd3427aa6cc643"
I20260812 06:51:48.628793 26307 raft_consensus.cc:3037] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:48.628880 26533 raft_consensus.cc:3037] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:48.632071 26533 raft_consensus.cc:2454] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab0e9e8612f4483b27cb8566cf379e4 in term 1.
I20260812 06:51:48.632082 26307 raft_consensus.cc:2454] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab0e9e8612f4483b27cb8566cf379e4 in term 1.
I20260812 06:51:48.632449 26650 leader_election.cc:302] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a9c3a506cf41efae45b8f60e223b7d, eab0e9e8612f4483b27cb8566cf379e4; no voters:
I20260812 06:51:48.632752 26904 raft_consensus.cc:2781] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:48.633184 26904 raft_consensus.cc:686] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 1 LEADER]: Becoming Leader. State: Replica: eab0e9e8612f4483b27cb8566cf379e4, State: Running, Role: LEADER
I20260812 06:51:48.633592 26904 consensus_queue.cc:237] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } }
I20260812 06:51:48.638070 26044 catalog_manager.cc:5495] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to eab0e9e8612f4483b27cb8566cf379e4 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "eab0e9e8612f4483b27cb8566cf379e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab0e9e8612f4483b27cb8566cf379e4" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 43803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 } health_report { overall_health: UNKNOWN } } }
I20260812 06:51:48.652655 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.650988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.17.69.67:43803)
0812 06:51:48.651123 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.651127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.651227 (+ 100us) server_negotiation.cc:407] Connection header received
0812 06:51:48.651385 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.651388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.651459 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.651533 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.652238 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.652243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.652246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.652260 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.652296 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.652307 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.652395 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.652443 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.652485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:51:48.660192 26533 raft_consensus.cc:1261] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 1 FOLLOWER]: Refusing update from remote peer eab0e9e8612f4483b27cb8566cf379e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:48.660205 26307 raft_consensus.cc:1261] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d [term 1 FOLLOWER]: Refusing update from remote peer eab0e9e8612f4483b27cb8566cf379e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:51:48.661079 26907 consensus_queue.cc:1035] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a9c3a506cf41efae45b8f60e223b7d" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:48.661401 26904 consensus_queue.cc:1035] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1386c7fe87eb4b8882dd3427aa6cc643" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:51:48.692138 26043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:48.692445 26043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:51:48.696401 26043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b473490534664d1880b307b576f0352f: Sending DeleteTablet for 3 replicas of tablet 5e13564ea8e44e27a53da3696b2daf1c
I20260812 06:51:48.697463 26728 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:48 UTC) from {username='slave'} at 127.0.0.1:46656
I20260812 06:51:48.697628 26264 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:48 UTC) from {username='slave'} at 127.0.0.1:55040
I20260812 06:51:48.698542 26506 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e13564ea8e44e27a53da3696b2daf1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:51:48 UTC) from {username='slave'} at 127.0.0.1:58606
I20260812 06:51:48.704409 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26143
I20260812 06:51:48.709554 26935 tablet_replica.cc:330] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: stopping tablet replica
I20260812 06:51:48.710044 26935 raft_consensus.cc:2229] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:51:48.710598 26935 raft_consensus.cc:2258] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:48.712767 26935 ts_tablet_manager.cc:1905] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:48.719846 26935 ts_tablet_manager.cc:1918] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:48.720052 26935 log.cc:1196] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/5e13564ea8e44e27a53da3696b2daf1c
I20260812 06:51:48.720595 26935 ts_tablet_manager.cc:1939] T 5e13564ea8e44e27a53da3696b2daf1c P 1386c7fe87eb4b8882dd3427aa6cc643: Deleting consensus metadata
I20260812 06:51:48.722496 26022 catalog_manager.cc:4841] TS 1386c7fe87eb4b8882dd3427aa6cc643 (127.17.69.66:36823): tablet 5e13564ea8e44e27a53da3696b2daf1c (table test-table [id=55078b04285448d8a0de231016b51c04]) successfully deleted
I20260812 06:51:48.723685 26933 tablet_replica.cc:330] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: stopping tablet replica
I20260812 06:51:48.724174 26933 raft_consensus.cc:2229] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:51:48.725072 26933 raft_consensus.cc:2258] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:51:48.725440 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.706782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.17.69.65:38495)
0812 06:51:48.706993 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.707001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.722787 (+ 15786us) server_negotiation.cc:407] Connection header received
0812 06:51:48.723198 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.723204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.723327 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.723427 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.724608 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.724616 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.724620 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.724642 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.724693 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.724710 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.724811 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.725065 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.725134 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:51:48.726665 26383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:48.727845 26933 ts_tablet_manager.cc:1905] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:51:48.735670 26933 ts_tablet_manager.cc:1918] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:51:48.736434 26933 log.cc:1196] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/5e13564ea8e44e27a53da3696b2daf1c
I20260812 06:51:48.737221 26933 ts_tablet_manager.cc:1939] T 5e13564ea8e44e27a53da3696b2daf1c P eab0e9e8612f4483b27cb8566cf379e4: Deleting consensus metadata
I20260812 06:51:48.739269 26019 catalog_manager.cc:4841] TS eab0e9e8612f4483b27cb8566cf379e4 (127.17.69.67:43803): tablet 5e13564ea8e44e27a53da3696b2daf1c (table test-table [id=55078b04285448d8a0de231016b51c04]) successfully deleted
I20260812 06:51:48.825338 26934 tablet_replica.cc:330] T 5e13564ea8e44e27a53da3696b2daf1c P a2a9c3a506cf41efae45b8f60e223b7d: stopping tablet replica
I20260812 06:51:48.825685 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26143
I20260812 06:51:48.854785 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26395
W20260812 06:51:48.854769 26019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.69.65:38495 (error 108)
I20260812 06:51:48.857435 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.856908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55066)
0812 06:51:48.857089 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.857299 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 06:51:48.857689 26019 catalog_manager.cc:4584] TS a2a9c3a506cf41efae45b8f60e223b7d (127.17.69.65:38495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e13564ea8e44e27a53da3696b2daf1c: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
I20260812 06:51:48.858165 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.855595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.17.69.66:36823)
0812 06:51:48.855799 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.855805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.855833 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:51:48.856088 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.856094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.856212 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.856321 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.857308 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.857315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.857320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.857341 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.857382 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.857398 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.857528 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.857857 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.857912 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:51:48.858860 26605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:48.911489 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.910958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55068)
0812 06:51:48.911188 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.911307 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 06:51:48.912618 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.912228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55076)
0812 06:51:48.912382 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.912470 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 06:51:48.944811 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26395
I20260812 06:51:48.968261 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26618
I20260812 06:51:48.971180 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.969114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.17.69.67:43803)
0812 06:51:48.969317 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:48.969323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:48.969372 (+ 49us) server_negotiation.cc:407] Connection header received
0812 06:51:48.969486 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:48.969491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:48.969556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:48.969666 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:48.970511 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:48.970518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:48.970522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:48.970543 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:48.970583 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:48.970598 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:48.970748 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:48.970806 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:51:48.970868 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:51:48.972018 26837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:48.973022 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.972492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55090)
0812 06:51:48.972726 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.972835 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 06:51:48.974073 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:48.973671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55106)
0812 06:51:48.973832 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:48.973922 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 06:51:49.054893 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:49.054440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55112)
0812 06:51:49.054651 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:49.054750 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 06:51:49.055912 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:49.055527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:38495 (local address 127.0.0.1:55116)
0812 06:51:49.055717 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:49.055790 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 06:51:49.092305 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26618
I20260812 06:51:49.120046 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25991
I20260812 06:51:49.121318 26087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:49.227046 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25991
2026-08-12T06:51:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2577 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 06:51:49.273684 17685 mini_oidc.cc:152] Starting JWKS server
I20260812 06:51:49.276598 17685 webserver.cc:460] Webserver started at https://127.0.0.1:41871/ using document root <none> and password file <none>
I20260812 06:51:49.276961 17685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:51:49.277740 17685 webserver.cc:460] Webserver started at http://127.0.0.1:45843/ using document root <none> and password file <none>
2026-08-12T06:51:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:49Z Disabled control of system clock
I20260812 06:51:49.311254 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46349
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:46349
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41871/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/testchainca.cert with env {}
W20260812 06:51:49.626523 26953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:49.626807 26953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:49.626942 26953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:49.633957 26953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:49.634074 26953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:49.634140 26953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:49.634219 26953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:49.634281 26953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:51:49.634383 26953 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:51:49.634440 26953 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41871/jwks.json
I20260812 06:51:49.644083 26953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:46349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41871/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:49.645241 26953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:49.646492 26953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:49.657526 26964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:49.657694 26966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:49.658066 26969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:49.658416 26953 server_base.cc:1034] running on GCE node
I20260812 06:51:49.659163 26953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:49.660519 26953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:49.661710 26953 hybrid_clock.cc:648] HybridClock initialized: now 1786517509661660 us; error 45 us; skew 500 ppm
I20260812 06:51:49.662042 26953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:49.664170 26953 webserver.cc:460] Webserver started at http://127.17.69.126:37343/ using document root <none> and password file <none>
I20260812 06:51:49.664799 26953 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:49.664898 26953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:49.665191 26953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:49.668012 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "c3de0366f14b41aa8c323268e30527a7"
format_stamp: "Formatted at 2026-08-12 06:51:49 on dist-test-slave-6zbq"
server_key: "e166b85f9a3aab962dd2561cbe084bb6"
server_key_iv: "f75292ca62d0011bc66e16c21ae66a78"
server_key_version: "encryptionkey@0"
I20260812 06:51:49.668788 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "c3de0366f14b41aa8c323268e30527a7"
format_stamp: "Formatted at 2026-08-12 06:51:49 on dist-test-slave-6zbq"
server_key: "e166b85f9a3aab962dd2561cbe084bb6"
server_key_iv: "f75292ca62d0011bc66e16c21ae66a78"
server_key_version: "encryptionkey@0"
I20260812 06:51:49.673441 26953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:51:49.676605 26975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:49.677824 26953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:49.677989 26953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "c3de0366f14b41aa8c323268e30527a7"
format_stamp: "Formatted at 2026-08-12 06:51:49 on dist-test-slave-6zbq"
server_key: "e166b85f9a3aab962dd2561cbe084bb6"
server_key_iv: "f75292ca62d0011bc66e16c21ae66a78"
server_key_version: "encryptionkey@0"
I20260812 06:51:49.678179 26953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:49.692088 26953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:49.692790 26953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:49.693053 26953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:49.710420 26953 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:46349
I20260812 06:51:49.710451 27064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:46349 every 8 connection(s)
I20260812 06:51:49.711661 26953 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:49.715358 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:49.720600 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26953
I20260812 06:51:49.720824 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:49.721626 17685 external_mini_cluster.cc:1428] Setting key cb4c9275b01081bc07f87c369422619c
I20260812 06:51:49.726370 27065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7: Bootstrap starting.
I20260812 06:51:49.729681 27065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:49.730908 27065 log.cc:824] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:49.733862 27065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7: No bootstrap required, opened a new log
I20260812 06:51:49.734930 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:49.723541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.17.69.126:46349)
0812 06:51:49.724610 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:49.724621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:49.724667 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:51:49.726289 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:49.726310 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:49.726794 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:49.727081 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:49.727710 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.728963 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.730805 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.731584 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.731793 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:49.732809 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:49.732817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:49.732829 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:49.733318 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:49.733391 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:49.733416 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:49.733498 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:49.733723 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 06:51:49.734002 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":428,"threads_started":1}
I20260812 06:51:49.738505 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:51:49.741861 27065 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } }
I20260812 06:51:49.742182 27065 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:49.742273 27065 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3de0366f14b41aa8c323268e30527a7, State: Initialized, Role: FOLLOWER
I20260812 06:51:49.742760 27065 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } }
I20260812 06:51:49.742967 27065 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:49.743083 27065 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:49.743247 27065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:49.745321 27065 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } }
I20260812 06:51:49.745818 27065 leader_election.cc:302] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3de0366f14b41aa8c323268e30527a7; no voters:
I20260812 06:51:49.746299 27065 leader_election.cc:288] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:49.746461 27073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:49.747130 27073 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [term 1 LEADER]: Becoming Leader. State: Replica: c3de0366f14b41aa8c323268e30527a7, State: Running, Role: LEADER
I20260812 06:51:49.747624 27073 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } }
I20260812 06:51:49.748157 27065 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:49.750365 27074 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3de0366f14b41aa8c323268e30527a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } } }
I20260812 06:51:49.750566 27077 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3de0366f14b41aa8c323268e30527a7. Latest consensus state: current_term: 1 leader_uuid: "c3de0366f14b41aa8c323268e30527a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3de0366f14b41aa8c323268e30527a7" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46349 } } }
I20260812 06:51:49.751012 27077 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:49.750772 27074 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:49.755120 27088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:49.760262 27088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:49.768486 27088 catalog_manager.cc:1347] Generated new cluster ID: f16b821d24254d0796da35ba36b39440
I20260812 06:51:49.768580 27088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:49.784219 27088 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:49.785982 27088 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:49.801667 27088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3de0366f14b41aa8c323268e30527a7: Generated new TSK 0
I20260812 06:51:49.802788 27088 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:51:49.876926 27116 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 06:51:49.877169 27116 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:51:49.878146 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:49.856457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.17.69.126:46349)
0812 06:51:49.856684 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:49.856691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:49.857359 (+ 668us) server_negotiation.cc:407] Connection header received
0812 06:51:49.857773 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:49.857781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:49.857869 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:49.858002 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:51:49.858666 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.859433 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.861478 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.861941 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.862046 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:49.862697 (+ 651us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:51:49.877633 (+ 14936us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:51:49.877925 (+ 292us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 06:51:49.878274 27071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 06:51:49.880396 27114 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:49.856331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:46349 (local address 127.0.0.1:33476)
0812 06:51:49.857229 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:49.857251 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:49.857464 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:49.858049 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:49.858054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:49.858069 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:51:49.858514 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:49.858528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.859581 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.859588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:49.861316 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:49.861333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:49.862122 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:49.862137 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:49.862308 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:49.862432 (+ 124us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:51:49.877918 (+ 15486us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 06:51:49.878284 (+ 366us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.69.126:46349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":496,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.69.126:46349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 06:51:49.882073 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26953
I20260812 06:51:49.883325 27060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:49.979276 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26953
2026-08-12T06:51:50Z chronyd exiting
I20260812 06:51:50.036515 17685 test_util.cc:175] -----------------------------------------------
I20260812 06:51:50.036667 17685 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786517466184829-17685-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 06:51:50.040479 17685 mini_oidc.cc:152] Starting JWKS server
I20260812 06:51:50.042274 17685 webserver.cc:460] Webserver started at https://127.0.0.1:35359/ using document root <none> and password file <none>
I20260812 06:51:50.042508 17685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:51:50.043120 17685 webserver.cc:460] Webserver started at http://127.0.0.1:43289/ using document root <none> and password file <none>
2026-08-12T06:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:50Z Disabled control of system clock
I20260812 06:51:50.075223 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:35531
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:34251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:35531
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35359/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/testchainca.cert with env {}
W20260812 06:51:50.381101 27128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:50.381389 27128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:50.381525 27128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:50.388434 27128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:50.388532 27128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:50.388597 27128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:50.388676 27128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:50.388744 27128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:51:50.388832 27128 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:51:50.388885 27128 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35359/jwks.json
I20260812 06:51:50.398353 27128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:34251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:35531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:35531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35359/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:50.399480 27128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:50.400681 27128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:50.414634 27136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:50.414665 27140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:50.414685 27134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:50.415783 27128 server_base.cc:1034] running on GCE node
I20260812 06:51:50.416607 27128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:50.418057 27128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:50.419263 27128 hybrid_clock.cc:648] HybridClock initialized: now 1786517510419222 us; error 31 us; skew 500 ppm
I20260812 06:51:50.419610 27128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:50.426312 27128 webserver.cc:460] Webserver started at http://127.17.69.126:42209/ using document root <none> and password file <none>
I20260812 06:51:50.427071 27128 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:50.427184 27128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:50.427510 27128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:50.430440 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "38c06bf8e9b344b9bf5a5134184a506e"
format_stamp: "Formatted at 2026-08-12 06:51:50 on dist-test-slave-6zbq"
server_key: "6376ef1b8af976fbbd66dd4ca8690219"
server_key_iv: "883cf393da6385870699664935c74488"
server_key_version: "encryptionkey@0"
I20260812 06:51:50.431227 27128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "38c06bf8e9b344b9bf5a5134184a506e"
format_stamp: "Formatted at 2026-08-12 06:51:50 on dist-test-slave-6zbq"
server_key: "6376ef1b8af976fbbd66dd4ca8690219"
server_key_iv: "883cf393da6385870699664935c74488"
server_key_version: "encryptionkey@0"
I20260812 06:51:50.436508 27128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:51:50.439914 27150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:50.441200 27128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:51:50.441370 27128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "38c06bf8e9b344b9bf5a5134184a506e"
format_stamp: "Formatted at 2026-08-12 06:51:50 on dist-test-slave-6zbq"
server_key: "6376ef1b8af976fbbd66dd4ca8690219"
server_key_iv: "883cf393da6385870699664935c74488"
server_key_version: "encryptionkey@0"
I20260812 06:51:50.441569 27128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:50.460076 27128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:50.460826 27128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:50.461113 27128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:50.477912 27128 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:35531
I20260812 06:51:50.477931 27246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:35531 every 8 connection(s)
I20260812 06:51:50.479115 27128 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:50.482362 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 27128
I20260812 06:51:50.482560 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:50.482679 27247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:50.483425 17685 external_mini_cluster.cc:1428] Setting key 495cc531a0d35cd1974cf76682432833
I20260812 06:51:50.494089 27247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e: Bootstrap starting.
I20260812 06:51:50.496483 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:50.485303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.17.69.126:35531)
0812 06:51:50.486267 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:50.486277 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:50.486318 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:51:50.488531 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:50.488552 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:50.488970 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:50.489321 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:50.489819 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.491085 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.492320 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.492934 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.493096 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:50.494246 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:50.494256 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:50.494269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:50.494796 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:50.494868 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:50.494889 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:50.494986 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:50.495220 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 06:51:50.495504 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":343,"threads_started":1}
I20260812 06:51:50.497222 27247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:50.498463 27247 log.cc:824] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:50.500486 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:51:50.501190 27247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e: No bootstrap required, opened a new log
I20260812 06:51:50.508134 27247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } }
I20260812 06:51:50.508447 27247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:50.508535 27247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c06bf8e9b344b9bf5a5134184a506e, State: Initialized, Role: FOLLOWER
I20260812 06:51:50.509012 27247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } }
I20260812 06:51:50.509251 27247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:50.509400 27247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:50.509554 27247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:50.511673 27247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } }
I20260812 06:51:50.512118 27247 leader_election.cc:302] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c06bf8e9b344b9bf5a5134184a506e; no voters:
I20260812 06:51:50.512607 27247 leader_election.cc:288] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:50.512739 27256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:50.513396 27256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [term 1 LEADER]: Becoming Leader. State: Replica: 38c06bf8e9b344b9bf5a5134184a506e, State: Running, Role: LEADER
I20260812 06:51:50.513881 27256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } }
I20260812 06:51:50.514390 27247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:50.516469 27258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c06bf8e9b344b9bf5a5134184a506e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } } }
I20260812 06:51:50.516867 27258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:50.517701 27273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:50.517567 27259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c06bf8e9b344b9bf5a5134184a506e. Latest consensus state: current_term: 1 leader_uuid: "38c06bf8e9b344b9bf5a5134184a506e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c06bf8e9b344b9bf5a5134184a506e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 35531 } } }
I20260812 06:51:50.518496 27259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:50.522845 27273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:50.531507 27273 catalog_manager.cc:1347] Generated new cluster ID: 373c700960a240ae8f0b58d5e7ed3e31
I20260812 06:51:50.531594 27273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:50.543454 27273 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:50.544576 27273 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:50.554665 27273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c06bf8e9b344b9bf5a5134184a506e: Generated new TSK 0
I20260812 06:51:50.555295 27273 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 06:51:50.586483 27303 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 06:51:50.586584 27303 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 06:51:50.587800 27302 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:50.568313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:35531 (local address 127.0.0.1:37010)
0812 06:51:50.568913 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:50.568928 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:50.569037 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:50.569439 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:50.569442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:50.569451 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:51:50.569709 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:50.569717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.571105 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.571109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:50.572141 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:50.572151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.573084 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.573088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:50.573217 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:50.573249 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:51:50.587348 (+ 14099us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:51:50.587529 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.69.126:35531: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":314,"threads_started":1}
I20260812 06:51:50.587834 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:50.568425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.17.69.126:35531)
0812 06:51:50.568606 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:50.568616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:50.569006 (+ 390us) server_negotiation.cc:407] Connection header received
0812 06:51:50.569177 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:50.569183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:50.569277 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:50.569435 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:51:50.569871 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.570980 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.572276 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:50.572947 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:50.573077 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:50.573498 (+ 421us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:51:50.587152 (+ 13654us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 06:51:50.587529 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 06:51:50.587949 27253 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 06:51:50.589187 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 27128
I20260812 06:51:50.590243 27239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:50.683508 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 27128
2026-08-12T06:51:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (707 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 06:51:50.747522 17685 mini_oidc.cc:152] Starting JWKS server
I20260812 06:51:50.749303 17685 webserver.cc:460] Webserver started at https://127.0.0.1:35111/ using document root <none> and password file <none>
I20260812 06:51:50.749526 17685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 06:51:50.750125 17685 webserver.cc:460] Webserver started at http://127.0.0.1:35071/ using document root <none> and password file <none>
2026-08-12T06:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:50Z Disabled control of system clock
I20260812 06:51:50.781786 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40651
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40651
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35111/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:51:51.089677 27323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:51.089962 27323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:51.090094 27323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:51.097018 27323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:51.097117 27323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:51.097182 27323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:51.097260 27323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:51.097324 27323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:51:51.097407 27323 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:51:51.097460 27323 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35111/jwks.json
I20260812 06:51:51.106029 27323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35111/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:51.107169 27323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:51.108842 27323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:51.119647 27331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:51.119592 27336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:51.119690 27338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:51.120812 27323 server_base.cc:1034] running on GCE node
I20260812 06:51:51.121544 27323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:51.122818 27323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:51.124017 27323 hybrid_clock.cc:648] HybridClock initialized: now 1786517511123969 us; error 38 us; skew 500 ppm
I20260812 06:51:51.124331 27323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:51.126381 27323 webserver.cc:460] Webserver started at http://127.17.69.126:46037/ using document root <none> and password file <none>
I20260812 06:51:51.127012 27323 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:51.127112 27323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:51.127416 27323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:51.130031 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "6454720e844b4ab98c91ad37c399d242"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "3bc332d459e16577964d0416682f9ced"
server_key_iv: "1496baba77c78b5ac148a66f85c9e133"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.130836 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "6454720e844b4ab98c91ad37c399d242"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "3bc332d459e16577964d0416682f9ced"
server_key_iv: "1496baba77c78b5ac148a66f85c9e133"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.135391 27323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:51:51.138523 27347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:51.139678 27323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:51:51.139827 27323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "6454720e844b4ab98c91ad37c399d242"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "3bc332d459e16577964d0416682f9ced"
server_key_iv: "1496baba77c78b5ac148a66f85c9e133"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.140014 27323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:51.175460 27323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:51.176225 27323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:51.176507 27323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:51.193971 27323 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40651
I20260812 06:51:51.194002 27431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40651 every 8 connection(s)
I20260812 06:51:51.195236 27323 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:51.198585 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 27323
I20260812 06:51:51.198833 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:51.198976 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:51.199687 17685 external_mini_cluster.cc:1428] Setting key 11e918fe73cb4f5dbc672e3c4205b6c7
I20260812 06:51:51.211697 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242: Bootstrap starting.
I20260812 06:51:51.213325 27438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:51.201485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47800 (local address 127.17.69.126:40651)
0812 06:51:51.202528 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:51.202538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:51.202582 (+ 44us) server_negotiation.cc:407] Connection header received
0812 06:51:51.204824 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:51.204845 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:51.205289 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:51.205603 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:51.206126 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.207369 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.209228 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.209908 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.210034 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:51.211215 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:51.211223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:51.211235 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:51.211724 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:51.211831 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:51.211856 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:51.211941 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:51.212180 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:51:51.212447 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":374,"threads_started":1}
I20260812 06:51:51.214717 27433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:51.215754 27433 log.cc:824] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:51.216785 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:51:51.218308 27433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242: No bootstrap required, opened a new log
I20260812 06:51:51.225868 27433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } }
I20260812 06:51:51.226173 27433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:51.226269 27433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6454720e844b4ab98c91ad37c399d242, State: Initialized, Role: FOLLOWER
I20260812 06:51:51.226788 27433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } }
I20260812 06:51:51.226989 27433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:51.227113 27433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:51.227252 27433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:51.229348 27433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } }
I20260812 06:51:51.229794 27433 leader_election.cc:302] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6454720e844b4ab98c91ad37c399d242; no voters:
I20260812 06:51:51.230320 27433 leader_election.cc:288] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:51.230454 27441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:51.231088 27441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [term 1 LEADER]: Becoming Leader. State: Replica: 6454720e844b4ab98c91ad37c399d242, State: Running, Role: LEADER
I20260812 06:51:51.231607 27441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } }
I20260812 06:51:51.232098 27433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:51.233865 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6454720e844b4ab98c91ad37c399d242. Latest consensus state: current_term: 1 leader_uuid: "6454720e844b4ab98c91ad37c399d242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } } }
I20260812 06:51:51.234160 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:51.234555 27444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6454720e844b4ab98c91ad37c399d242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6454720e844b4ab98c91ad37c399d242" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40651 } } }
I20260812 06:51:51.234941 27444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:51.236346 27448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:51.241452 27448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:51.249200 27448 catalog_manager.cc:1347] Generated new cluster ID: 2fb11eb3ba0348e0b99ac6afd93d9fc5
I20260812 06:51:51.249300 27448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:51.288852 27448 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:51.290009 27448 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:51.295567 27448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6454720e844b4ab98c91ad37c399d242: Generated new TSK 0
I20260812 06:51:51.296267 27448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:51:51.324867 27438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:51.317211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.17.69.126:40651)
0812 06:51:51.317408 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:51.317415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:51.317860 (+ 445us) server_negotiation.cc:407] Connection header received
0812 06:51:51.318049 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:51.318053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:51.318116 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:51.318232 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:51:51.318764 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.319449 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.321401 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.321841 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.321932 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:51.322550 (+ 618us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:51:51.324283 (+ 1733us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:51:51.324581 (+ 298us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 06:51:51.325099 27490 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:51.317180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40651 (local address 127.0.0.1:47808)
0812 06:51:51.317756 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:51.317779 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:51.317920 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:51.318257 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:51.318262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:51.318275 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:51:51.318620 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:51.318632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.319575 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.319581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:51.321266 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:51.321283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.322015 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.322022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:51.322250 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:51.322296 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:51:51.324468 (+ 2172us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 06:51:51.324696 (+ 228us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.69.126:40651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":301,"threads_started":1}
W20260812 06:51:51.325229 27438 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260812 06:51:51.326227 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 27323
I20260812 06:51:51.327319 27426 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 06:51:51.341769 27491 webserver.cc:533] Webserver: sslize failed:
E20260812 06:51:51.341874 27491 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 06:51:51.423198 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 27323
2026-08-12T06:51:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (743 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T06:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:51Z Disabled control of system clock
I20260812 06:51:51.520505 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38615
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38615
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/testchainca.cert with env {}
W20260812 06:51:51.830024 27505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:51.830308 27505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:51.830436 27505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:51.837359 27505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:51.837446 27505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:51.837507 27505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:51.837579 27505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:51.837641 27505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 06:51:51.837723 27505 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 06:51:51.837777 27505 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 06:51:51.847213 27505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:51.848304 27505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:51.849455 27505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:51.862140 27513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:51.862269 27514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:51.862195 27518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:51.863132 27505 server_base.cc:1034] running on GCE node
I20260812 06:51:51.863763 27505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:51.864953 27505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:51.866150 27505 hybrid_clock.cc:648] HybridClock initialized: now 1786517511866109 us; error 34 us; skew 500 ppm
I20260812 06:51:51.866488 27505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:51:51.868579 27505 webserver.cc:460] Webserver started at http://127.17.69.126:40919/ using document root <none> and password file <none>
I20260812 06:51:51.869196 27505 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:51.869302 27505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:51.869578 27505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:51.872264 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "0c60dfa074a24e92816fc241ed099fe0"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "a23627e15a813dfc28b89332b563c621"
server_key_iv: "14dbae541cd77db18b3f626f667bb203"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.872995 27505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "0c60dfa074a24e92816fc241ed099fe0"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "a23627e15a813dfc28b89332b563c621"
server_key_iv: "14dbae541cd77db18b3f626f667bb203"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.877569 27505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:51:51.880650 27527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:51.881799 27505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:51:51.881955 27505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "0c60dfa074a24e92816fc241ed099fe0"
format_stamp: "Formatted at 2026-08-12 06:51:51 on dist-test-slave-6zbq"
server_key: "a23627e15a813dfc28b89332b563c621"
server_key_iv: "14dbae541cd77db18b3f626f667bb203"
server_key_version: "encryptionkey@0"
I20260812 06:51:51.882158 27505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:51.898737 27505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:51.899428 27505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:51.899674 27505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:51.915966 27505 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38615
I20260812 06:51:51.915997 27610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38615 every 8 connection(s)
I20260812 06:51:51.917111 27505 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:51.917847 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 27505
I20260812 06:51:51.918056 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:51.918764 17685 external_mini_cluster.cc:1428] Setting key 881c0dcb70ab17d60292b9189f49ec0b
I20260812 06:51:51.920995 27611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:51.931216 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:51.920401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.17.69.126:38615)
0812 06:51:51.921334 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:51.921344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:51.921384 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:51:51.922931 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:51.922952 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:51.923370 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:51.923669 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:51.924228 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.925475 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.927204 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:51.927804 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:51.927935 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:51.929104 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:51.929111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:51.929123 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:51:51.929628 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:51.929700 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:51:51.929723 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:51.929807 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:51.930021 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 06:51:51.930387 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":319,"threads_started":1}
I20260812 06:51:51.932505 27611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0: Bootstrap starting.
I20260812 06:51:51.934741 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:51:51.935487 27611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:51.936380 27611 log.cc:824] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:51.939005 27611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0: No bootstrap required, opened a new log
I20260812 06:51:51.945842 27611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } }
I20260812 06:51:51.946180 27611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:51.946265 27611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c60dfa074a24e92816fc241ed099fe0, State: Initialized, Role: FOLLOWER
I20260812 06:51:51.946765 27611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } }
I20260812 06:51:51.946987 27611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:51.947105 27611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:51.947252 27611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:51.949191 27611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } }
I20260812 06:51:51.949637 27611 leader_election.cc:302] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c60dfa074a24e92816fc241ed099fe0; no voters:
I20260812 06:51:51.950173 27611 leader_election.cc:288] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:51.950294 27624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:51.950958 27624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [term 1 LEADER]: Becoming Leader. State: Replica: 0c60dfa074a24e92816fc241ed099fe0, State: Running, Role: LEADER
I20260812 06:51:51.951517 27624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } }
I20260812 06:51:51.952188 27611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:51.954499 27625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c60dfa074a24e92816fc241ed099fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } } }
I20260812 06:51:51.954452 27628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c60dfa074a24e92816fc241ed099fe0. Latest consensus state: current_term: 1 leader_uuid: "0c60dfa074a24e92816fc241ed099fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c60dfa074a24e92816fc241ed099fe0" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38615 } } }
I20260812 06:51:51.954859 27625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:51.954859 27628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:51.955703 27637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:51.960422 27637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:51.968899 27637 catalog_manager.cc:1347] Generated new cluster ID: 0f293cb181f346c5b91a98eedadc7d2e
I20260812 06:51:51.968997 27637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:51.987687 27637 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:51.988807 27637 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:52.004737 27637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c60dfa074a24e92816fc241ed099fe0: Generated new TSK 0
I20260812 06:51:52.005702 27637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:51:52.141106 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:52.028697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.17.69.126:38615)
0812 06:51:52.028880 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:52.028887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:52.029369 (+ 482us) server_negotiation.cc:407] Connection header received
0812 06:51:52.029581 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:52.029584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:52.029660 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:52.029769 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0812 06:51:52.030310 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:52.031201 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:52.033149 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:52.033581 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:52.033668 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:52.034186 (+ 518us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 06:51:52.140523 (+106337us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:51:52.140857 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50756: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:51:52.141243 27620 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50756: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 06:51:52.141582 27673 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:52.028668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38615 (local address 127.0.0.1:50756)
0812 06:51:52.029266 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:52.029286 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:52.029431 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:52.029805 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:52.029809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:52.029823 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0812 06:51:52.030117 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:52.030157 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:52.031356 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:52.031362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:52.033018 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:52.033034 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:52.033739 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:52.033746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:52.033922 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:52.033967 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 06:51:52.140838 (+106871us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 06:51:52.141175 (+ 337us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.69.126:38615: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":301,"threads_started":1}
I20260812 06:51:52.142974 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 27505
I20260812 06:51:52.144178 27603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:52.245491 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 27505
2026-08-12T06:51:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (795 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (399 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (399 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:53 dist-test-slave-6zbq krb5kdc[27695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:53 dist-test-slave-6zbq krb5kdc[27695](info): set up 2 sockets
Aug 12 06:51:53 dist-test-slave-6zbq krb5kdc[27695](info): commencing operation
krb5kdc: starting...
W20260812 06:51:54.280102 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:54 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:51:54.488619 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:39057
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:54.799615 27720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:54.799896 27720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:54.800031 27720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:54.807199 27720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:54.807315 27720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:54.807390 27720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:54.807467 27720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:54.807528 27720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:54.817121 27720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:54.818392 27720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:54.819640 27720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:54.830765 27731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:54.830804 27734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:54.831272 27739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:54.831578 27720 server_base.cc:1034] running on GCE node
I20260812 06:51:54.832255 27720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:54.833495 27720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:54.834697 27720 hybrid_clock.cc:648] HybridClock initialized: now 1786517514834648 us; error 29 us; skew 500 ppm
I20260812 06:51:54.835003 27720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:54 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517514, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:54.837893 27720 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:51:54.839329 27720 webserver.cc:460] Webserver started at http://127.17.69.126:41325/ using document root <none> and password file <none>
I20260812 06:51:54.839953 27720 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:54.840063 27720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:54.840363 27720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:54.843217 27720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "5e6079bb4f4449c2a88a4be26f64a6e6"
format_stamp: "Formatted at 2026-08-12 06:51:54 on dist-test-slave-6zbq"
server_key: "d157b91033aea22ab35858eee2fe3c8a"
server_key_iv: "5e65c644f5c1e7204de9b9f650e96b32"
server_key_version: "encryptionkey@0"
I20260812 06:51:54.843966 27720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "5e6079bb4f4449c2a88a4be26f64a6e6"
format_stamp: "Formatted at 2026-08-12 06:51:54 on dist-test-slave-6zbq"
server_key: "d157b91033aea22ab35858eee2fe3c8a"
server_key_iv: "5e65c644f5c1e7204de9b9f650e96b32"
server_key_version: "encryptionkey@0"
I20260812 06:51:54.848807 27720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:54.852047 27762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:54.853259 27720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:51:54.853415 27720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "5e6079bb4f4449c2a88a4be26f64a6e6"
format_stamp: "Formatted at 2026-08-12 06:51:54 on dist-test-slave-6zbq"
server_key: "d157b91033aea22ab35858eee2fe3c8a"
server_key_iv: "5e65c644f5c1e7204de9b9f650e96b32"
server_key_version: "encryptionkey@0"
I20260812 06:51:54.853586 27720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:54.867156 27720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:54.871078 27720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:54.871327 27720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:54.888098 27720 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:39057
I20260812 06:51:54.888142 27848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:39057 every 8 connection(s)
I20260812 06:51:54.889300 27720 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:54.892719 27856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:54.895987 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 27720
I20260812 06:51:54.896186 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:54.896814 17685 external_mini_cluster.cc:1428] Setting key fb7d933a19848800997272c4c8d416a0
I20260812 06:51:54.905278 27856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6: Bootstrap starting.
I20260812 06:51:54.908614 27856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:54.909540 27856 log.cc:824] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:51:54 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:54.912324 27856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6: No bootstrap required, opened a new log
I20260812 06:51:54.919560 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:54.898663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.17.69.126:39057)
0812 06:51:54.899460 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:54.899476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:54.899527 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:51:54.901225 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:54.901253 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:54.901730 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:54.902142 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:54.903465 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:54.905372 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:54.906636 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:54.907648 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:54.907825 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:54.911557 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:54.911578 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:54.911596 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:54.911637 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:54.915417 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:54.916041 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:54.916049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:54.916060 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:54.916263 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:54.916574 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:54.916579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:54.916586 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:54.917234 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:54.917544 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:54.917839 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 06:51:54.918252 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":334,"threads_started":1}
I20260812 06:51:54.922443 27856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } }
I20260812 06:51:54.922773 27856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:54.922848 27856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6079bb4f4449c2a88a4be26f64a6e6, State: Initialized, Role: FOLLOWER
I20260812 06:51:54.923327 27856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } }
I20260812 06:51:54.923528 27856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:54.923645 27856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:54.923779 27856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:54.926016 27856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } }
I20260812 06:51:54.926496 27856 leader_election.cc:302] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e6079bb4f4449c2a88a4be26f64a6e6; no voters:
I20260812 06:51:54.927011 27856 leader_election.cc:288] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:54.927150 27864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:54.927772 27864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6079bb4f4449c2a88a4be26f64a6e6, State: Running, Role: LEADER
I20260812 06:51:54.928368 27864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } }
I20260812 06:51:54.928975 27856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:54.931329 27869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e6079bb4f4449c2a88a4be26f64a6e6. Latest consensus state: current_term: 1 leader_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } } }
I20260812 06:51:54.931301 27867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6079bb4f4449c2a88a4be26f64a6e6" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39057 } } }
I20260812 06:51:54.931790 27869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:54.931821 27867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:54.933522 27881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:54.938115 27881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:54.945588 27881 catalog_manager.cc:1347] Generated new cluster ID: 48240daaaaaa441092977ed50f490b80
I20260812 06:51:54.945752 27881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:54.976275 27881 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:54.977559 27881 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:54.985736 27881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e6079bb4f4449c2a88a4be26f64a6e6: Generated new TSK 0
I20260812 06:51:54.986557 27881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:51:55.051200 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:55.369612 27897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:55.369890 27897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:55.370030 27897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:55.377166 27897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:55.377278 27897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:55.377430 27897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:55.386760 27897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:55.387938 27897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:55.389175 27897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:55.401407 27910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:55.401486 27905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:55.401443 27906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:55.402225 27897 server_base.cc:1034] running on GCE node
I20260812 06:51:55.402799 27897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:55.403623 27897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:55.404821 27897 hybrid_clock.cc:648] HybridClock initialized: now 1786517515404776 us; error 36 us; skew 500 ppm
I20260812 06:51:55.405103 27897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:55 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:55.408013 27897 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:51:55.409296 27897 webserver.cc:460] Webserver started at http://127.17.69.65:33129/ using document root <none> and password file <none>
I20260812 06:51:55.409919 27897 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:55.410039 27897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:55.410532 27897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:55.413297 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "da585de1a6b9465d91228a0798bc6e45"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "3289fa48ee8d610243c286b540253920"
server_key_iv: "2a0a849821807ca8cc95a176a00d4589"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.414026 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "da585de1a6b9465d91228a0798bc6e45"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "3289fa48ee8d610243c286b540253920"
server_key_iv: "2a0a849821807ca8cc95a176a00d4589"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.418788 27897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:51:55.421936 27918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.423120 27897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:51:55.423274 27897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "da585de1a6b9465d91228a0798bc6e45"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "3289fa48ee8d610243c286b540253920"
server_key_iv: "2a0a849821807ca8cc95a176a00d4589"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.423445 27897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:55.440955 27897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:55.444722 27897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:55.444965 27897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:55.445812 27897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:55.447017 27897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:55.447088 27897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.447196 27897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:55.447247 27897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.476567 27897 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:35641
I20260812 06:51:55.476619 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:35641 every 8 connection(s)
I20260812 06:51:55.477742 27897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:55.481539 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 27897
I20260812 06:51:55.481760 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:55.482607 17685 external_mini_cluster.cc:1428] Setting key 18a3d062c4a74b2869e8ac9f6a0f130a
Aug 12 06:51:55 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:55.498306 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:55.480115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:56403 (local address 127.17.69.126:39057)
0812 06:51:55.480553 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:55.480558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:55.481573 (+ 1015us) server_negotiation.cc:407] Connection header received
0812 06:51:55.483919 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:55.483925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:55.484010 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:55.484130 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:55.485973 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.486729 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.488734 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.489149 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.489243 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:55.492979 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:55.492984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:55.492988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:55.493019 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:55.495457 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:55.496102 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:55.496108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:55.496112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:55.496271 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:55.496646 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:55.496650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:55.496653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:55.496963 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:55.497111 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:55.497746 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 06:51:55.497957 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":212,"threads_started":1}
I20260812 06:51:55.498980 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:55.480598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39057 (local address 127.17.69.65:56403)
0812 06:51:55.481388 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:55.481437 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:55.483646 (+ 2209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:55.484309 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:55.484319 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:55.484981 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:55.485776 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:55.485801 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.486881 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.486887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:55.488594 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:55.488610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.489357 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.489364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:55.489621 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:55.490266 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:55.490291 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:55.492779 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:55.495587 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:55.495593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:55.495603 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:55.495984 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:55.496367 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:55.496370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:55.496372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:55.496549 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:55.497104 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:55.497110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:55.497389 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 06:51:55.497603 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 06:51:55.497953 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":266,"threads_started":1}
I20260812 06:51:55.500484 28125 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39057
I20260812 06:51:55.500751 28125 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:55.501425 28125 heartbeater.cc:511] Master 127.17.69.126:39057 requested a full tablet report, sending...
I20260812 06:51:55.503403 27789 ts_manager.cc:194] Registered new tserver with Master: da585de1a6b9465d91228a0798bc6e45 (127.17.69.65:35641)
I20260812 06:51:55.504817 27789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:56403
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:51:55.535081 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:55.844702 28137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:55.844955 28137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:55.845093 28137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:55.852033 28137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:55.852135 28137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:55.852284 28137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:55.861553 28137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:55.862722 28137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:55.863925 28137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:55.875936 28148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:55.875977 28149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:55.876020 28154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:55.876973 28137 server_base.cc:1034] running on GCE node
I20260812 06:51:55.877533 28137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:55.878391 28137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:55.879596 28137 hybrid_clock.cc:648] HybridClock initialized: now 1786517515879540 us; error 45 us; skew 500 ppm
I20260812 06:51:55.879868 28137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:55 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:55.882727 28137 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:51:55.883996 28137 webserver.cc:460] Webserver started at http://127.17.69.66:39033/ using document root <none> and password file <none>
I20260812 06:51:55.884595 28137 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:55.884719 28137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:55.884968 28137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:55.887843 28137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "01111be059ec4cb09bf813c279b055f8"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "ef186e879d2af96720554a5d26be0776"
server_key_iv: "0f07883bc69d1b89c4afc1427d10295b"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.888803 28137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "01111be059ec4cb09bf813c279b055f8"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "ef186e879d2af96720554a5d26be0776"
server_key_iv: "0f07883bc69d1b89c4afc1427d10295b"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.894773 28137 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 06:51:55.898670 28165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.900293 28137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:51:55.900468 28137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "01111be059ec4cb09bf813c279b055f8"
format_stamp: "Formatted at 2026-08-12 06:51:55 on dist-test-slave-6zbq"
server_key: "ef186e879d2af96720554a5d26be0776"
server_key_iv: "0f07883bc69d1b89c4afc1427d10295b"
server_key_version: "encryptionkey@0"
I20260812 06:51:55.900691 28137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:55.925125 28137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:55.929009 28137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:55.929256 28137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:55.930120 28137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:55.931345 28137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:55.931421 28137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.931535 28137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:55.931586 28137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:55.959373 28137 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:37861
I20260812 06:51:55.959414 28367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:37861 every 8 connection(s)
I20260812 06:51:55.960536 28137 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:55.964715 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 28137
I20260812 06:51:55.964905 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:55.965318 17685 external_mini_cluster.cc:1428] Setting key c53244adb700d34d0a7f60770c942d5c
Aug 12 06:51:55 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:55.979565 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:55.962957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:52605 (local address 127.17.69.126:39057)
0812 06:51:55.963135 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:55.963140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:55.964280 (+ 1140us) server_negotiation.cc:407] Connection header received
0812 06:51:55.966167 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:55.966173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:55.966264 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:55.966370 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:55.967895 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.968602 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.970052 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.970503 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.970611 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:55.973883 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:55.973888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:55.973893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:55.973928 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:55.976656 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:55.977469 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:55.977473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:55.977475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:55.977594 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:55.978011 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:55.978014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:55.978016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:55.978274 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:55.978421 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:55.979112 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 06:51:55.979326 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:51:55.980697 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:55.963432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39057 (local address 127.17.69.66:52605)
0812 06:51:55.964125 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:55.964165 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:55.965892 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:55.966597 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:55.966605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:55.967207 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:55.967739 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:55.967756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.968751 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.968755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:55.969917 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:55.969929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:55.970728 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:55.970738 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:55.970932 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:55.971526 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:55.971546 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:55.973667 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:55.976791 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:55.976800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:55.976813 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:55.977371 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:55.977692 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:55.977696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:55.977699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:55.977918 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:55.978413 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:55.978420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:55.978771 (+ 351us) client_negotiation.cc:769] Sending connection context
0812 06:51:55.979040 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 06:51:55.979500 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":283,"threads_started":1}
I20260812 06:51:55.981981 28369 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39057
I20260812 06:51:55.982260 28369 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:55.982935 28369 heartbeater.cc:511] Master 127.17.69.126:39057 requested a full tablet report, sending...
I20260812 06:51:55.984305 27789 ts_manager.cc:194] Registered new tserver with Master: 01111be059ec4cb09bf813c279b055f8 (127.17.69.66:37861)
I20260812 06:51:55.985080 27789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:52605
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:51:56.015395 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:56.327198 28377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:56.327450 28377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:56.327585 28377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:56.334640 28377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:56.334744 28377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:56.334888 28377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:51:56.344034 28377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:56.345212 28377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:56.346460 28377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:56.358805 28391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:56.358845 28390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:56.359339 28395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:56.360162 28377 server_base.cc:1034] running on GCE node
I20260812 06:51:56.360692 28377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:56.361577 28377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:56.362783 28377 hybrid_clock.cc:648] HybridClock initialized: now 1786517516362737 us; error 40 us; skew 500 ppm
I20260812 06:51:56.363032 28377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.365826 28377 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:51:56.367118 28377 webserver.cc:460] Webserver started at http://127.17.69.67:35603/ using document root <none> and password file <none>
I20260812 06:51:56.367733 28377 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:56.367842 28377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:56.368124 28377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:56.370868 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "24930d7dbd3146358b8a016e5b5f3fff"
format_stamp: "Formatted at 2026-08-12 06:51:56 on dist-test-slave-6zbq"
server_key: "666d651e5864679bc596d5cdc4ac165c"
server_key_iv: "cc8dde1b5149cc03b555b9c9e35505a6"
server_key_version: "encryptionkey@0"
I20260812 06:51:56.371614 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "24930d7dbd3146358b8a016e5b5f3fff"
format_stamp: "Formatted at 2026-08-12 06:51:56 on dist-test-slave-6zbq"
server_key: "666d651e5864679bc596d5cdc4ac165c"
server_key_iv: "cc8dde1b5149cc03b555b9c9e35505a6"
server_key_version: "encryptionkey@0"
I20260812 06:51:56.376334 28377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:51:56.379463 28414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:56.380788 28377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:51:56.380939 28377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "24930d7dbd3146358b8a016e5b5f3fff"
format_stamp: "Formatted at 2026-08-12 06:51:56 on dist-test-slave-6zbq"
server_key: "666d651e5864679bc596d5cdc4ac165c"
server_key_iv: "cc8dde1b5149cc03b555b9c9e35505a6"
server_key_version: "encryptionkey@0"
I20260812 06:51:56.381219 28377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:56.402427 28377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:56.406071 28377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:56.406303 28377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:56.407203 28377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:56.408360 28377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:56.408440 28377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:56.408555 28377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:56.408607 28377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:56.437569 28377 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:33411
I20260812 06:51:56.437628 28623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:33411 every 8 connection(s)
I20260812 06:51:56.438730 28377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:51:56.443938 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 28377
I20260812 06:51:56.444119 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:51:56.444538 17685 external_mini_cluster.cc:1428] Setting key 4c474f34724e4db1efbcffe7ee863c76
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:51:56.456822 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.440823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:34363 (local address 127.17.69.126:39057)
0812 06:51:56.441029 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.441035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.442192 (+ 1157us) server_negotiation.cc:407] Connection header received
0812 06:51:56.444042 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.444048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.444133 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.444244 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.445875 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.446745 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.448181 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.448575 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.448694 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.451725 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.451730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.451733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.451766 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.454016 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.454710 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.454714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.454717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.454835 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.455222 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.455226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.455228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.455478 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.455609 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.456228 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.456444 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:51:56.457535 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.441326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39057 (local address 127.17.69.67:34363)
0812 06:51:56.442025 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:56.442065 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:56.443760 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:56.444556 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:56.444563 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:56.445052 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:56.445706 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:56.445737 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.446880 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.446886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:56.448041 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:56.448053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.448782 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.448790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:56.449021 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.449538 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:56.449559 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:56.451528 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:56.454142 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:56.454150 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:56.454164 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:56.454595 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:56.454936 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:56.454940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:56.454944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:56.455109 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:56.455600 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:56.455607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:56.455873 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 06:51:56.456083 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 06:51:56.456521 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":278,"threads_started":1}
I20260812 06:51:56.458825 28624 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39057
I20260812 06:51:56.459098 28624 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:56.459743 28624 heartbeater.cc:511] Master 127.17.69.126:39057 requested a full tablet report, sending...
I20260812 06:51:56.461011 27788 ts_manager.cc:194] Registered new tserver with Master: 24930d7dbd3146358b8a016e5b5f3fff (127.17.69.67:33411)
I20260812 06:51:56.461884 27788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:34363
I20260812 06:51:56.471123 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/test-admin.keytab.
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.500321 17685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 06:51:56.504947 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:56.502837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.17.69.65:35641)
0812 06:51:56.503427 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.503434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.503477 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:51:56.503534 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.503540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.503732 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.503845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.504585 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51412: BlockingRecv error: recv got EOF from 127.0.0.1:51412 (error 108)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":300,"threads_started":1}
W20260812 06:51:56.505039 17685 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.506619 17685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.507550 28125 heartbeater.cc:503] Master 127.17.69.126:39057 was elected leader, sending a full tablet report...
I20260812 06:51:56.507997 17685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:56.522253 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.509871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.17.69.65:35641)
0812 06:51:56.510009 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.510013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.510360 (+ 347us) server_negotiation.cc:407] Connection header received
0812 06:51:56.510520 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.510524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.510582 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.510653 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.511576 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.512524 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.513622 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.514138 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.514233 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.517483 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.517494 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.517507 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.517536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.520072 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.520664 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.520669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.520672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.520791 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.521057 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.521060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.521062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.521395 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.521603 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.521836 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.522016 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 06:51:56.527187 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:56.525739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.17.69.65:35641)
0812 06:51:56.525912 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.525917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.526122 (+ 205us) server_negotiation.cc:407] Connection header received
0812 06:51:56.526225 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.526230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.526310 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.526451 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.526996 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51418: BlockingRecv error: recv got EOF from 127.0.0.1:51418 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 06:51:56.527446 17685 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.528806 17685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.529999 17685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:56.541929 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.531094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.17.69.65:35641)
0812 06:51:56.531249 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.531256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.531533 (+ 277us) server_negotiation.cc:407] Connection header received
0812 06:51:56.531641 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.531644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.531702 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.531764 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.532659 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.533333 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.534449 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.534840 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.534923 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.537896 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.537901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.537904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.537931 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.540050 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.540566 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.540569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.540571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.540671 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.540948 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.540951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.540952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.541184 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.541367 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.541538 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.541689 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 06:51:56.545470 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:51:56.544061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.17.69.65:35641)
0812 06:51:56.544221 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.544226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.544356 (+ 130us) server_negotiation.cc:407] Connection header received
0812 06:51:56.544489 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.544494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.544558 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.544633 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.545280 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51442: BlockingRecv error: recv got EOF from 127.0.0.1:51442 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:51:56.545572 17685 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786517466184829-17685-0/krb5kdc/krb5cc' of type FILE
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.546890 17685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:56.547966 17685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:51:56.560550 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.549008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.17.69.65:35641)
0812 06:51:56.549169 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.549174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.549432 (+ 258us) server_negotiation.cc:407] Connection header received
0812 06:51:56.549555 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.549559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.549607 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.549679 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.550727 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.551384 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.552537 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.552946 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.553049 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.556142 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.556146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.556149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.556177 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.558478 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.559105 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.559108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.559111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.559226 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.559551 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.559555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.559557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.559805 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.560009 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.560186 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.560326 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:51:56.561895 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 27897
I20260812 06:51:56.572283 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.562490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.17.69.65:35641)
0812 06:51:56.562600 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.562605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.562936 (+ 331us) server_negotiation.cc:407] Connection header received
0812 06:51:56.563059 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.563064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.563115 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.563187 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.564033 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.564641 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.565777 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.566152 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.566265 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.568021 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.568026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.568029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.568056 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.569992 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.570626 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.570631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.570635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.570769 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.571094 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.571098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.571100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.571322 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.571544 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.571867 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.572043 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 06:51:56.572715 28117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:56.657927 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 27897
I20260812 06:51:56.679455 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 28137
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:51:56.694175 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.680263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.17.69.66:37861)
0812 06:51:56.680788 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.680795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.680822 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:51:56.680884 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.680890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.681111 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.681260 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.682198 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.683251 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.684515 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.685100 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.685203 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.688979 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.688993 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.689007 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.689049 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.691804 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.692436 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.692442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.692445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.692558 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.692935 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.692938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.692940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.693258 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.693483 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.693692 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.693891 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":257,"threads_started":1}
I20260812 06:51:56.695528 28359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:56.775310 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 28137
I20260812 06:51:56.796070 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 28377
Aug 12 06:51:56 dist-test-slave-6zbq krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:51:56.810493 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:56.797011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.17.69.67:33411)
0812 06:51:56.797618 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:56.797625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:56.797652 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:51:56.797722 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:56.797729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:56.797931 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:56.798052 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:56.799235 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.800246 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.801418 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:56.801957 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:56.802050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:56.805566 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:56.805578 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:56.805590 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:56.805625 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:56.808262 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.808791 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.808797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.808799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.808916 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:56.809201 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:56.809204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:56.809206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:56.809512 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:56.809729 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:56.809928 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:51:56.810160 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":303,"threads_started":1}
I20260812 06:51:56.811652 28613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:56.891629 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 28377
I20260812 06:51:56.911788 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 27720
I20260812 06:51:56.912878 27843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:51:57.012874 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 27720
2026-08-12T06:51:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3969 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:51:57 dist-test-slave-6zbq krb5kdc[28674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:51:57 dist-test-slave-6zbq krb5kdc[28674](info): set up 2 sockets
Aug 12 06:51:57 dist-test-slave-6zbq krb5kdc[28674](info): commencing operation
krb5kdc: starting...
W20260812 06:51:58.237654 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:51:58 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:51:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:51:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab.
Entry for principal oryx/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab.
I20260812 06:51:58.463359 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45281
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab
--principal=oryx/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:58.776352 28695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:58.776628 28695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:58.776717 28695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:58.783694 28695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:51:58.783790 28695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:58.783846 28695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:58.783902 28695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:51:58.783948 28695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:51:58.793258 28695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab
--principal=oryx/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:58.794592 28695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:58.795797 28695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:58.806646 28709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:58.806694 28705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:58.806591 28704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:58.808059 28695 server_base.cc:1034] running on GCE node
I20260812 06:51:58.808799 28695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:58.810104 28695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:58.811309 28695 hybrid_clock.cc:648] HybridClock initialized: now 1786517518811252 us; error 47 us; skew 500 ppm
I20260812 06:51:58.811616 28695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:58 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:58.814513 28695 init.cc:377] Logged in from keytab as oryx/127.17.69.126@KRBTEST.COM (short username oryx)
I20260812 06:51:58.815832 28695 webserver.cc:460] Webserver started at http://127.17.69.126:37161/ using document root <none> and password file <none>
I20260812 06:51:58.816439 28695 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:58.816556 28695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:58.816839 28695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:58.819662 28695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "cc9679c7494c476ca100b4489a0c9886"
format_stamp: "Formatted at 2026-08-12 06:51:58 on dist-test-slave-6zbq"
server_key: "cc2b1e7b2bb29a6fa372cc3c0665520d"
server_key_iv: "10efd8de9c722e370f6d5753f87c4a59"
server_key_version: "encryptionkey@0"
I20260812 06:51:58.820389 28695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "cc9679c7494c476ca100b4489a0c9886"
format_stamp: "Formatted at 2026-08-12 06:51:58 on dist-test-slave-6zbq"
server_key: "cc2b1e7b2bb29a6fa372cc3c0665520d"
server_key_iv: "10efd8de9c722e370f6d5753f87c4a59"
server_key_version: "encryptionkey@0"
I20260812 06:51:58.825074 28695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:51:58.828156 28717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:58.829309 28695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:58.829466 28695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "cc9679c7494c476ca100b4489a0c9886"
format_stamp: "Formatted at 2026-08-12 06:51:58 on dist-test-slave-6zbq"
server_key: "cc2b1e7b2bb29a6fa372cc3c0665520d"
server_key_iv: "10efd8de9c722e370f6d5753f87c4a59"
server_key_version: "encryptionkey@0"
I20260812 06:51:58.829638 28695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:58.845852 28695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:58.849180 28695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:58.849416 28695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:58.865286 28695 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:45281
I20260812 06:51:58.865322 28795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:45281 every 8 connection(s)
I20260812 06:51:58.866540 28695 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:51:58.869767 28798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:51:58.871361 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 28695
I20260812 06:51:58.871578 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:51:58.872411 17685 external_mini_cluster.cc:1428] Setting key e60134510198b0458958e6162c4f7827
I20260812 06:51:58.883116 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Bootstrap starting.
Aug 12 06:51:58 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:51:58.886468 28798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Neither blocks nor log segments found. Creating new log.
I20260812 06:51:58.887603 28798 log.cc:824] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Log is configured to *not* fsync() on all Append() calls
I20260812 06:51:58.890684 28798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: No bootstrap required, opened a new log
I20260812 06:51:58.894815 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:58.874310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.17.69.126:45281)
0812 06:51:58.875059 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:58.875072 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:58.875115 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:51:58.876606 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:58.876628 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:58.876968 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:58.877244 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:58.879016 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:58.880258 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:58.882040 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:58.882696 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:58.882821 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:58.887017 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:58.887028 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:58.887043 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:58.887090 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:58.890649 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:58.891335 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:58.891343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:58.891351 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:58.891538 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:58.891867 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:58.891872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:58.891875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:58.892523 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:58.892861 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:58.893160 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 06:51:58.893550 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":338,"threads_started":1}
I20260812 06:51:58.899497 28798 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } }
I20260812 06:51:58.899796 28798 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:51:58.899866 28798 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9679c7494c476ca100b4489a0c9886, State: Initialized, Role: FOLLOWER
I20260812 06:51:58.900338 28798 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } }
I20260812 06:51:58.900528 28798 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:51:58.900622 28798 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:51:58.900745 28798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:51:58.902925 28798 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } }
I20260812 06:51:58.903344 28798 leader_election.cc:302] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc9679c7494c476ca100b4489a0c9886; no voters:
I20260812 06:51:58.903859 28798 leader_election.cc:288] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:51:58.903961 28814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:51:58.904561 28814 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 LEADER]: Becoming Leader. State: Replica: cc9679c7494c476ca100b4489a0c9886, State: Running, Role: LEADER
I20260812 06:51:58.905031 28814 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } }
I20260812 06:51:58.905727 28798 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:51:58.907831 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc9679c7494c476ca100b4489a0c9886. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } }
I20260812 06:51:58.907814 28815 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } }
I20260812 06:51:58.908269 28815 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:58.908269 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:51:58.909202 28835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:51:58.914727 28835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:51:58.922847 28835 catalog_manager.cc:1347] Generated new cluster ID: db88e73cc6b743f68c1fe768ac5d5bc2
I20260812 06:51:58.922940 28835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:51:58.942641 28835 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:51:58.944283 28835 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:51:58.955766 28835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Generated new TSK 0
I20260812 06:51:58.956418 28835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
I20260812 06:51:59.026489 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:59.345618 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:59.345858 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:59.345985 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:59.353042 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:59.353147 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:59.353281 28862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:51:59.362407 28862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:59.363495 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:59.364712 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:59.376924 28877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:59.376967 28878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:59.377008 28883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:59.377734 28862 server_base.cc:1034] running on GCE node
I20260812 06:51:59.378422 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:59.379244 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:59.380435 28862 hybrid_clock.cc:648] HybridClock initialized: now 1786517519380399 us; error 28 us; skew 500 ppm
I20260812 06:51:59.380702 28862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:59 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:59.383628 28862 init.cc:377] Logged in from keytab as oryx/127.17.69.65@KRBTEST.COM (short username oryx)
I20260812 06:51:59.384917 28862 webserver.cc:460] Webserver started at http://127.17.69.65:38167/ using document root <none> and password file <none>
I20260812 06:51:59.385499 28862 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:59.385596 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:59.385833 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:59.388540 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "973e161db5644dd2b5c260e6129d7f2a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "09e9451791af2bb830d5c92120bd72a8"
server_key_iv: "90953853067ea3ddef84815a9829c2ae"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.389281 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "973e161db5644dd2b5c260e6129d7f2a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "09e9451791af2bb830d5c92120bd72a8"
server_key_iv: "90953853067ea3ddef84815a9829c2ae"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.394004 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 06:51:59.397162 28899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.398300 28862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:51:59.398475 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "973e161db5644dd2b5c260e6129d7f2a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "09e9451791af2bb830d5c92120bd72a8"
server_key_iv: "90953853067ea3ddef84815a9829c2ae"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.398648 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:59.420362 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:59.424115 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:59.424363 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:59.425201 28862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:59.426411 28862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:59.426482 28862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.426595 28862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:59.426646 28862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.456561 28862 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:46729
I20260812 06:51:59.456626 29103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:46729 every 8 connection(s)
I20260812 06:51:59.457741 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:51:59.459106 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 28862
I20260812 06:51:59.459324 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:51:59.460083 17685 external_mini_cluster.cc:1428] Setting key 23c36f3dbb8501921affe30b0a975882
Aug 12 06:51:59 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:51:59.476830 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:59.460297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:40045 (local address 127.17.69.126:45281)
0812 06:51:59.460733 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:59.460740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:59.461815 (+ 1075us) server_negotiation.cc:407] Connection header received
0812 06:51:59.463641 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:59.463647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:59.463722 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:59.463840 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:59.465187 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.466076 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.467631 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.468179 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.468304 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:59.471604 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:59.471609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:59.471612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:59.471642 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:59.474012 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:59.474722 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:59.474726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:59.474728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:59.474846 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:59.475239 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:59.475243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:59.475245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:59.475489 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:59.475620 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:59.476264 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 06:51:59.476483 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":202,"threads_started":1}
I20260812 06:51:59.477655 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:59.460819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:40045)
0812 06:51:59.461665 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:59.461716 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:59.463388 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:59.464014 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:59.464021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:59.464482 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:59.465025 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:59.465043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.466205 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.466208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:59.467488 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:59.467501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.468369 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.468375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:59.468561 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:59.469162 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:59.469184 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:59.471402 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:59.474155 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:59.474161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:59.474171 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:59.474603 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:59.474950 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:59.474954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:59.474956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:59.475126 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:59.475627 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:59.475635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:59.475906 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 06:51:59.476120 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 06:51:59.476569 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":360,"threads_started":1}
I20260812 06:51:59.479135 29106 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:51:59.479453 29106 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:59.480278 29106 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:51:59.482424 28745 ts_manager.cc:194] Registered new tserver with Master: 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729)
I20260812 06:51:59.483799 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:40045
WARNING: no policy specified for oryx/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
I20260812 06:51:59.511294 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:51:59.821300 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:51:59.821553 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:51:59.821684 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:51:59.828572 29117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:51:59.828676 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:51:59.828821 29117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:51:59.838027 29117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:51:59.839159 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:51:59.840337 29117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:51:59.852213 29127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:59.852217 29130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:51:59.852417 29124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:51:59.852892 29117 server_base.cc:1034] running on GCE node
I20260812 06:51:59.853545 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:51:59.854429 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:51:59.855616 29117 hybrid_clock.cc:648] HybridClock initialized: now 1786517519855568 us; error 39 us; skew 500 ppm
I20260812 06:51:59.855895 29117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:51:59 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:51:59.858788 29117 init.cc:377] Logged in from keytab as oryx/127.17.69.66@KRBTEST.COM (short username oryx)
I20260812 06:51:59.860018 29117 webserver.cc:460] Webserver started at http://127.17.69.66:42243/ using document root <none> and password file <none>
I20260812 06:51:59.860608 29117 fs_manager.cc:359] Metadata directory not provided
I20260812 06:51:59.860719 29117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:51:59.860985 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:51:59.863686 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "ef735495ae43443aa42cca77b43ea95a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "995178fd3a0db466ea83f75b2776d412"
server_key_iv: "84959b4cc51493f866b94da0c7b4436d"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.864396 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "ef735495ae43443aa42cca77b43ea95a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "995178fd3a0db466ea83f75b2776d412"
server_key_iv: "84959b4cc51493f866b94da0c7b4436d"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.869362 29117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:51:59.872653 29142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.873871 29117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:51:59.874017 29117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "ef735495ae43443aa42cca77b43ea95a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "995178fd3a0db466ea83f75b2776d412"
server_key_iv: "84959b4cc51493f866b94da0c7b4436d"
server_key_version: "encryptionkey@0"
I20260812 06:51:59.874192 29117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:51:59.899515 29117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:51:59.903483 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:51:59.903734 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:51:59.904565 29117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:51:59.905840 29117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:51:59.905912 29117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.906025 29117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:51:59.906077 29117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:51:59.934015 29117 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:36363
I20260812 06:51:59.934065 29348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:36363 every 8 connection(s)
I20260812 06:51:59.935194 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:51:59.939657 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 29117
I20260812 06:51:59.939812 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:51:59.940238 17685 external_mini_cluster.cc:1428] Setting key b37b52d710279e4cc0a9dd710d5cfe38
Aug 12 06:51:59 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:51:59.954815 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:59.937458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58247 (local address 127.17.69.126:45281)
0812 06:51:59.937678 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 06:51:59.937683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:51:59.938740 (+ 1057us) server_negotiation.cc:407] Connection header received
0812 06:51:59.940546 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:51:59.940552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:51:59.940629 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:51:59.940725 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:51:59.942333 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.943076 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.945129 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.945515 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.945601 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:59.949132 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:51:59.949139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:51:59.949143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:51:59.949185 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:51:59.951967 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:59.952618 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:59.952622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:59.952624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:59.952772 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:51:59.953169 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:51:59.953173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:51:59.953175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:51:59.953463 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:51:59.953597 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:51:59.954221 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 06:51:59.954471 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 06:51:59.955623 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:51:59.937886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:58247)
0812 06:51:59.938601 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 06:51:59.938636 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:51:59.940274 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:51:59.940940 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:51:59.940951 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:51:59.941503 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:51:59.942155 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:59.942180 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.943239 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.943244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:59.945005 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:51:59.945019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:51:59.945720 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:51:59.945727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:51:59.945978 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:51:59.946632 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:51:59.946657 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:51:59.948918 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:51:59.952087 (+ 3169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:59.952093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:59.952103 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:59.952501 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:59.952878 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:51:59.952882 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:51:59.952884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:51:59.953060 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:51:59.953593 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:51:59.953600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:51:59.953875 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 06:51:59.954098 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 06:51:59.954592 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":286,"threads_started":1}
I20260812 06:51:59.956948 29349 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:51:59.957216 29349 heartbeater.cc:464] Registering TS with master...
I20260812 06:51:59.957844 29349 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:51:59.959223 28745 ts_manager.cc:194] Registered new tserver with Master: ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:51:59.959981 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:58247
WARNING: no policy specified for oryx/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
I20260812 06:51:59.988842 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:00.301148 29355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:00.301416 29355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:00.301515 29355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:00.308501 29355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:00.308600 29355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:00.308722 29355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:00.317926 29355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:00.319027 29355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:00.320217 29355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:00.333459 29373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:00.333508 29369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:00.333714 29368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:00.334111 29355 server_base.cc:1034] running on GCE node
I20260812 06:52:00.334762 29355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:00.335572 29355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:00.336776 29355 hybrid_clock.cc:648] HybridClock initialized: now 1786517520336736 us; error 36 us; skew 500 ppm
I20260812 06:52:00.337042 29355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:00.339970 29355 init.cc:377] Logged in from keytab as oryx/127.17.69.67@KRBTEST.COM (short username oryx)
I20260812 06:52:00.341279 29355 webserver.cc:460] Webserver started at http://127.17.69.67:42277/ using document root <none> and password file <none>
I20260812 06:52:00.341867 29355 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:00.341976 29355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:00.342244 29355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:00.344929 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "02b692dd8547f8c2a38ebe5d0550e7d6"
server_key_iv: "ba644d32add0128f760a3ae826c52285"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.345615 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "02b692dd8547f8c2a38ebe5d0550e7d6"
server_key_iv: "ba644d32add0128f760a3ae826c52285"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.350311 29355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 06:52:00.353436 29382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:00.354632 29355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 06:52:00.354769 29355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "02b692dd8547f8c2a38ebe5d0550e7d6"
server_key_iv: "ba644d32add0128f760a3ae826c52285"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.354939 29355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:00.368412 29355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:00.371920 29355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:00.372171 29355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:00.373062 29355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:00.374300 29355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:00.374440 29355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:00.374540 29355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:00.374588 29355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:00.403977 29355 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41185
I20260812 06:52:00.404021 29592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41185 every 8 connection(s)
I20260812 06:52:00.405220 29355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:00.406940 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 29355
I20260812 06:52:00.407119 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:00.407621 17685 external_mini_cluster.cc:1428] Setting key 289cb8f7af6dd2e889a494772f7acdfc
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:00.423583 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.407827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:54425 (local address 127.17.69.126:45281)
0812 06:52:00.408040 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.408045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.409311 (+ 1266us) server_negotiation.cc:407] Connection header received
0812 06:52:00.411168 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.411172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.411238 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.411318 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.412618 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.413366 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.414832 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.415226 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.415320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.418272 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.418277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.418280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.418312 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.420562 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.421305 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.421311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.421315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.421440 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.421995 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.422000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.422004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.422362 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.422513 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.423186 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.423343 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:52:00.424304 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.408350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:54425)
0812 06:52:00.409121 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.409183 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.410939 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.411490 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.411498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.411949 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.412478 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.412494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.413483 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.413487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.414718 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.414731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.415383 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.415387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.415564 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.416097 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.416118 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:00.418099 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:00.420709 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.420718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.420730 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.421201 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.421643 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.421647 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.421650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.421888 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.422549 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:00.422556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:00.422870 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 06:52:00.423077 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:52:00.423399 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":308,"threads_started":1}
I20260812 06:52:00.425415 29593 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:52:00.425681 29593 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:00.426390 29593 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:52:00.427662 28745 ts_manager.cc:194] Registered new tserver with Master: 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
I20260812 06:52:00.428395 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:54425
I20260812 06:52:00.434652 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.69.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.125.keytab.
Entry for principal oryx/127.17.69.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.125.keytab.
I20260812 06:52:00.483589 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:36565
--webserver_interface=127.17.69.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.125.keytab
--principal=oryx/127.17.69.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45281,127.17.69.125:36565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:52:00.486410 29106 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
W20260812 06:52:00.793897 29600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:00.794138 29600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:00.794273 29600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:00.801167 29600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:00.801275 29600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:00.801334 29600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:00.801414 29600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:00.801473 29600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:00.811081 29600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45281,127.17.69.125:36565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.125.keytab
--principal=oryx/127.17.69.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:36565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.69.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
W20260812 06:52:00.811369 29600 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.69.126:45281,127.17.69.125:36565'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 06:52:00.812314 29600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:00.813494 29600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:00.824572 29614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:00.824625 29611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:00.824647 29612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:00.827423 29600 server_base.cc:1034] running on GCE node
I20260812 06:52:00.828076 29600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:00.829298 29600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:00.830497 29600 hybrid_clock.cc:648] HybridClock initialized: now 1786517520830438 us; error 39 us; skew 500 ppm
I20260812 06:52:00.830783 29600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:00.837253 29600 init.cc:377] Logged in from keytab as oryx/127.17.69.125@KRBTEST.COM (short username oryx)
I20260812 06:52:00.838611 29600 webserver.cc:460] Webserver started at http://127.17.69.125:45975/ using document root <none> and password file <none>
I20260812 06:52:00.839282 29600 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:00.839367 29600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:00.839623 29600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:00.842409 29600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/instance:
uuid: "283f83cd6ce041b49d02336f4773bf4a"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "af82624897046471ac70efe1706e5688"
server_key_iv: "53bd37b87e742e34ed7db42fa2e7fa80"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.843133 29600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal/instance:
uuid: "283f83cd6ce041b49d02336f4773bf4a"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "af82624897046471ac70efe1706e5688"
server_key_iv: "53bd37b87e742e34ed7db42fa2e7fa80"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.847817 29600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:52:00.850919 29623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:00.852097 29600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:52:00.852238 29600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "283f83cd6ce041b49d02336f4773bf4a"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "af82624897046471ac70efe1706e5688"
server_key_iv: "53bd37b87e742e34ed7db42fa2e7fa80"
server_key_version: "encryptionkey@0"
I20260812 06:52:00.852406 29600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:00.864730 29600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:00.868371 29600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:00.868590 29600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:00.885254 29600 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.125:36565
I20260812 06:52:00.885272 29723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.125:36565 every 8 connection(s)
I20260812 06:52:00.886484 29600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
I20260812 06:52:00.889707 29725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:00.891355 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 29600
I20260812 06:52:00.898185 29725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.125@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:00.913244 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.893399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49050 (local address 127.17.69.125:36565)
0812 06:52:00.894211 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.894223 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.894261 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:52:00.895795 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.895820 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.896159 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.896443 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.898189 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.899530 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.901019 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.902096 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.902300 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.906580 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.906591 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.906605 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.906659 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.909685 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.910271 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.910278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.910284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.910460 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.910827 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.910832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.910836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.911293 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.911549 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.911993 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.912349 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":389,"threads_started":1}
I20260812 06:52:00.914826 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.900721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.17.69.126:45281)
0812 06:52:00.900929 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.900936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.901550 (+ 614us) server_negotiation.cc:407] Connection header received
0812 06:52:00.901731 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.901735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.901814 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.901893 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.903171 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.904269 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.905551 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.906016 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.906127 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.909837 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.909842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.909845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.909876 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.912329 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.913006 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.913012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.913015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.913143 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.913541 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.913544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.913546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.913847 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.913977 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.914466 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.914617 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:52:00.915093 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.900683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35818)
0812 06:52:00.901409 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.901449 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.901608 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.901979 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.902011 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.902752 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.903002 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.903012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.904396 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.904400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.905426 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.905438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.906225 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.906232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.906427 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.907040 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.907073 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:00.909663 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:00.912450 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.912461 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.912468 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.912894 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.913254 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.913259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.913262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.913432 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.913972 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:00.913980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:00.914150 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 06:52:00.914398 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 06:52:00.914700 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":345,"threads_started":1}
I20260812 06:52:00.916682 29725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517520, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.125@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
I20260812 06:52:00.927374 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.917208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49062)
0812 06:52:00.917366 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.917380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.917458 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.917664 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.917667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.917954 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.918071 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.918078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.918957 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.918960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.919769 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.919780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.920432 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.920435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.920577 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.920974 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.920988 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:00.922752 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:00.925281 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.925286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.925290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.925615 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.925911 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.925916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.925920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.926036 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.926548 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:00.926552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:00.926688 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 06:52:00.926799 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 06:52:00.927052 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 06:52:00.927506 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.917252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.17.69.125:36565)
0812 06:52:00.917384 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.917388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.917417 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:52:00.917521 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.917524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.917571 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.917643 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.918174 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.918845 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.919889 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.920309 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.920410 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.922907 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.922911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.922914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.922940 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.925166 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.925715 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.925720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.925722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.925819 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.926129 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.926134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.926137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.926440 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.926567 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.926895 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.927194 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":44}
I20260812 06:52:00.931419 29725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Bootstrap starting.
I20260812 06:52:00.934386 29725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:00.935391 29725 log.cc:824] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:00.937803 29725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: No bootstrap required, opened a new log
I20260812 06:52:00.944381 29725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } }
I20260812 06:52:00.944655 29725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:00.944747 29725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283f83cd6ce041b49d02336f4773bf4a, State: Initialized, Role: FOLLOWER
I20260812 06:52:00.945232 29725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } }
I20260812 06:52:00.945959 29734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } } }
I20260812 06:52:00.946269 29734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:00.946811 29725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:52:00.954777 29759 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:52:00.954881 29759 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:52:00.958406 29600 master_runner.cc:428] Detected that this master 283f83cd6ce041b49d02336f4773bf4a is joining an existing cluster
I20260812 06:52:00.958498 29600 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.69.125:36565
I20260812 06:52:00.962276 29349 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
I20260812 06:52:00.978926 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.962268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35826 (local address 127.17.69.126:45281)
0812 06:52:00.962479 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.962484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.963639 (+ 1155us) server_negotiation.cc:407] Connection header received
0812 06:52:00.964084 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.964089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.964189 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.964280 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.965960 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.967053 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.969019 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.969635 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.969754 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.972270 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.972275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.972279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.972313 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.975642 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.976448 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.976453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.976456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.976602 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.977022 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.977078 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.977082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.977402 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.977529 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.978320 (+ 791us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.978575 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:52:00.979637 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.962287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35826)
0812 06:52:00.963553 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.963572 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.963914 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.964460 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.964465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.965391 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.965792 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.965805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.967204 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.967209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.968849 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.968864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.969966 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.969972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.970136 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.970947 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.970969 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:00.972059 (+ 1090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:00.975797 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.975801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.975804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.976312 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.976734 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.976738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.976741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.976906 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.977588 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:00.977593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:00.977799 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 06:52:00.977944 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 06:52:00.979226 (+ 1282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":50,"thread_start_us":413,"threads_started":1}
I20260812 06:52:00.979769 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.963247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.17.69.125:36565)
0812 06:52:00.963594 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.963598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.964336 (+ 738us) server_negotiation.cc:407] Connection header received
0812 06:52:00.964468 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.964473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.964526 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.964606 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.965810 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.966909 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.968974 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.969675 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.969782 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.972157 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:00.972164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:00.972168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:00.972213 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:00.975626 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.976518 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.976523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.976526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.976673 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:00.977089 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:00.977093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:00.977096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:00.977443 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:00.977575 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:00.978055 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 06:52:00.979413 (+ 1358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1107,"server-negotiator.queue_time_us":53}
I20260812 06:52:00.980513 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.963158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49074)
0812 06:52:00.964253 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.964266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.964372 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.964632 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.964636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.965350 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.965669 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.965680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.967093 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.967099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.968828 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.968848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.970072 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.970079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.970244 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.970804 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.970827 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:00.971995 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:00.975769 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.975774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.975778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.976399 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.976818 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:00.976821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:00.976825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:00.976984 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:00.977658 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:00.977662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:00.977815 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 06:52:00.978147 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 06:52:00.980085 (+ 1938us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"mutex_wait_us":1701,"thread_start_us":823,"threads_started":1}
WARNING: no policy specified for oryx/127.17.69.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.124@KRBTEST.COM" created.
Aug 12 06:52:00 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.126@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
I20260812 06:52:01.007076 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.986271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.17.69.125:36565)
0812 06:52:00.986455 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:00.986461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:00.991055 (+ 4594us) server_negotiation.cc:407] Connection header received
0812 06:52:00.991800 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:00.991806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:00.991919 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:00.992021 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:00.993406 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.994578 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.996148 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.996833 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.996961 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.000705 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.000718 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.000723 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.000758 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.004016 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.004879 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.004884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.004888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.005040 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.005466 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.005471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.005473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.005804 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.005937 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.006497 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.006707 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 06:52:01.007151 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:00.986121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49088)
0812 06:52:00.990913 (+ 4792us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:00.990961 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:00.991529 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:00.992099 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:00.992108 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:00.992960 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:00.993230 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.993244 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.994734 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.994739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.996006 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:00.996019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:00.996987 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:00.996992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:00.997125 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:00.997752 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:00.997773 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.000496 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.004156 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.004169 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.004176 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.004747 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.005161 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.005166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.005170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.005345 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.005932 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.005940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.006139 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.006388 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.006713 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4534,"thread_start_us":4376,"threads_started":1}
I20260812 06:52:01.009642 28745 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.69.125:36565: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a"
member_type: NON_VOTER
last_known_addr {
host: "127.17.69.125"
port: 36565
}
attrs {
promote: true
}
}
dest_uuid: "cc9679c7494c476ca100b4489a0c9886"
cas_config_opid_index: -1
I20260812 06:52:01.011296 28745 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } }
I20260812 06:52:01.022094 29679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:01.021667 29782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 283f83cd6ce041b49d02336f4773bf4a (127.17.69.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } }
I20260812 06:52:01.025782 29679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 06:52:01.026087 29782 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.69.125:36565
I20260812 06:52:01.026407 29734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:01.026619 29782 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } }
I20260812 06:52:01.026677 29734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } }
I20260812 06:52:01.026851 29782 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:01.027304 29600 master.cc:537] Master@127.17.69.125:36565 shutting down...
I20260812 06:52:01.027143 29779 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 06:52:01.027974 29734 leader_election.cc:288] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281)
I20260812 06:52:01.029363 28760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "283f83cd6ce041b49d02336f4773bf4a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9679c7494c476ca100b4489a0c9886" is_pre_election: true
W20260812 06:52:01.030992 28726 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 06:52:01.031145 29630 leader_election.cc:302] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 283f83cd6ce041b49d02336f4773bf4a; no voters: cc9679c7494c476ca100b4489a0c9886
I20260812 06:52:01.033931 29734 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 06:52:01.034632 28726 consensus_peers.cc:487] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 -> Peer 283f83cd6ce041b49d02336f4773bf4a (127.17.69.125:36565): Couldn't send request to peer 283f83cd6ce041b49d02336f4773bf4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.124.keytab.
Entry for principal oryx/127.17.69.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.124.keytab.
I20260812 06:52:01.039541 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:36451
--webserver_interface=127.17.69.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.124.keytab
--principal=oryx/127.17.69.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:52:01.045667 29600 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:01.046042 29600 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:01.046227 29600 tablet_replica.cc:330] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: stopping tablet replica
I20260812 06:52:01.052990 29600 master.cc:552] Master@127.17.69.125:36565 shutdown complete.
I20260812 06:52:01.053100 29600 master_runner.cc:305] Clearing existing system tablet
I20260812 06:52:01.053484 29600 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:01.053574 29600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:01.055778 29600 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 06:52:01.057050 29787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:01.057328 29600 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:01.057593 29600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 06:52:01.057684 29600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "283f83cd6ce041b49d02336f4773bf4a"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "af82624897046471ac70efe1706e5688"
server_key_iv: "53bd37b87e742e34ed7db42fa2e7fa80"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.061731 29600 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:01.066037 29600 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:52:01.066181 29600 log.cc:1196] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 06:52:01.066581 29600 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Deleting consensus metadata
I20260812 06:52:01.066825 29600 master_runner.cc:315] Copying system tablet from 127.17.69.126:45281
I20260812 06:52:01.069134 29600 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: tablet copy: Beginning tablet copy session from remote peer at address 127.17.69.126:45281
I20260812 06:52:01.081454 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.070185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.17.69.126:45281)
0812 06:52:01.070581 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.070588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.071047 (+ 459us) server_negotiation.cc:407] Connection header received
0812 06:52:01.071231 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.071237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.071318 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.071400 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.072237 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.072930 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.074096 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.074555 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.074663 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.076302 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.076308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.076312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.076343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.078997 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.079580 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.079586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.079590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.079753 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.080128 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.080134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.080137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.080395 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.080524 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.080886 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.081111 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260812 06:52:01.081552 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.070052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35838)
0812 06:52:01.070945 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.070961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.071106 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.071446 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.071450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.071849 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.072079 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.072088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.073064 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.073070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.073955 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.073967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.074690 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.074696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.074878 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.075433 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.075449 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.076117 (+ 668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.079115 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.079120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.079125 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.079470 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.079866 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.079871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.079875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.080028 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.080509 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.080514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.080661 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.080783 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.081103 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":684,"threads_started":1}
I20260812 06:52:01.082588 28772 tablet_copy_service.cc:140] P cc9679c7494c476ca100b4489a0c9886: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 283f83cd6ce041b49d02336f4773bf4a ({username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:35838)
I20260812 06:52:01.082967 28772 tablet_copy_service.cc:161] P cc9679c7494c476ca100b4489a0c9886: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 283f83cd6ce041b49d02336f4773bf4a at {username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:35838: session id = 283f83cd6ce041b49d02336f4773bf4a-00000000000000000000000000000000
I20260812 06:52:01.086180 28772 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:52:01.088255 29600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:52:01.092780 29600 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: tablet copy: Starting download of 0 data blocks...
I20260812 06:52:01.093086 29600 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: tablet copy: Starting download of 1 WAL segments...
I20260812 06:52:01.095995 29600 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:52:01.099071 28772 tablet_copy_service.cc:342] P cc9679c7494c476ca100b4489a0c9886: Request end of tablet copy session 283f83cd6ce041b49d02336f4773bf4a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:35838
I20260812 06:52:01.099248 28772 tablet_copy_service.cc:433] P cc9679c7494c476ca100b4489a0c9886: ending tablet copy session 283f83cd6ce041b49d02336f4773bf4a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 283f83cd6ce041b49d02336f4773bf4a
I20260812 06:52:01.110693 29600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:01.113499 29801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:01.113999 29802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:01.120590 29600 server_base.cc:1034] running on GCE node
W20260812 06:52:01.121042 29806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:01.121519 29600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:01.122148 29600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:01.123363 29600 hybrid_clock.cc:648] HybridClock initialized: now 1786517521123292 us; error 37 us; skew 500 ppm
I20260812 06:52:01.123600 29600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:01.125648 29600 init.cc:377] Logged in from keytab as oryx/127.17.69.125@KRBTEST.COM (short username oryx)
I20260812 06:52:01.126868 29600 webserver.cc:460] Webserver started at http://127.17.69.125:42823/ using document root <none> and password file <none>
I20260812 06:52:01.127221 29600 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:01.127300 29600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:01.130277 29600 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:52:01.132320 29814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:01.132855 29600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 06:52:01.132972 29600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "283f83cd6ce041b49d02336f4773bf4a"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "af82624897046471ac70efe1706e5688"
server_key_iv: "53bd37b87e742e34ed7db42fa2e7fa80"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.133126 29600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:01.154294 29600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:01.158169 29600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:01.180343 29600 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.125:36565
I20260812 06:52:01.180380 29909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.125:36565 every 8 connection(s)
I20260812 06:52:01.181474 29600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
I20260812 06:52:01.186721 29910 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:52:01.190760 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Bootstrap starting.
I20260812 06:52:01.208527 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:52:01.209046 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Bootstrap complete.
I20260812 06:52:01.210111 29910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } }
I20260812 06:52:01.210390 29910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 283f83cd6ce041b49d02336f4773bf4a, State: Initialized, Role: LEARNER
I20260812 06:52:01.210623 29910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } }
I20260812 06:52:01.211685 29910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:01.211422 29914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } }
I20260812 06:52:01.211961 29914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: LEARNER
I20260812 06:52:01.223523 29940 catalog_manager.cc:1259] Loaded cluster ID: db88e73cc6b743f68c1fe768ac5d5bc2
I20260812 06:52:01.223603 29940 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: loading cluster ID for follower catalog manager: success
I20260812 06:52:01.226699 29940 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: acquiring CA information for follower catalog manager: success
I20260812 06:52:01.228878 29940 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:52:01.412477 29785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:01.412735 29785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:01.412863 29785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:01.420126 29785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:01.420218 29785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:01.420288 29785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:01.420359 29785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:01.420418 29785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:01.429643 29785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.124.keytab
--principal=oryx/127.17.69.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:36451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.69.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:01.430543 29593 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
I20260812 06:52:01.430749 29785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:01.432118 29785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:01.442840 29947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:01.442914 29950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:01.442878 29948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:01.443845 29785 server_base.cc:1034] running on GCE node
I20260812 06:52:01.444535 29785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:01.445683 29785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:01.446868 29785 hybrid_clock.cc:648] HybridClock initialized: now 1786517521446833 us; error 22 us; skew 500 ppm
I20260812 06:52:01.447170 29785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:01.449949 29785 init.cc:377] Logged in from keytab as oryx/127.17.69.124@KRBTEST.COM (short username oryx)
I20260812 06:52:01.451314 29785 webserver.cc:460] Webserver started at http://127.17.69.124:43905/ using document root <none> and password file <none>
I20260812 06:52:01.451875 29785 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:01.451973 29785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:01.452217 29785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:01.454995 29785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/instance:
uuid: "382f5faf48224ee1b4880786d7532819"
format_stamp: "Formatted at 2026-08-12 06:52:01 on dist-test-slave-6zbq"
server_key: "6b2c5c473355d90b10c34974ca4c84f0"
server_key_iv: "3dc07f7e90e73d59fd25c31a20b53aea"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.455777 29785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal/instance:
uuid: "382f5faf48224ee1b4880786d7532819"
format_stamp: "Formatted at 2026-08-12 06:52:01 on dist-test-slave-6zbq"
server_key: "6b2c5c473355d90b10c34974ca4c84f0"
server_key_iv: "3dc07f7e90e73d59fd25c31a20b53aea"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.460682 29785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:52:01.463846 29961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:01.465023 29785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:52:01.465165 29785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "382f5faf48224ee1b4880786d7532819"
format_stamp: "Formatted at 2026-08-12 06:52:01 on dist-test-slave-6zbq"
server_key: "6b2c5c473355d90b10c34974ca4c84f0"
server_key_iv: "3dc07f7e90e73d59fd25c31a20b53aea"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.465339 29785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:01.494627 29785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:01.499428 29785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:01.499719 29785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:01.521522 29785 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.124:36451
I20260812 06:52:01.521520 30071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.124:36451 every 8 connection(s)
I20260812 06:52:01.523263 29785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
I20260812 06:52:01.526717 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 29785
I20260812 06:52:01.526979 30072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:01.535820 30072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.124@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:01.550930 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.538174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35848 (local address 127.17.69.126:45281)
0812 06:52:01.538382 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.538388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.538802 (+ 414us) server_negotiation.cc:407] Connection header received
0812 06:52:01.538970 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.538976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.539047 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.539149 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.540192 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.540886 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.542328 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.542764 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.542875 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.545951 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.545955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.545958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.545990 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.548311 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.548945 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.548949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.548951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.549070 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.549442 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.549448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.549450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.549697 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.549826 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.550337 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.550601 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:52:01.551721 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.538009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35848)
0812 06:52:01.538662 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.538708 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.538859 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.539227 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.539232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.539808 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.540039 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.540051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.541022 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.541026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.542172 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.542185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.543060 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.543067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.543201 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.543736 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.543756 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.545776 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.548430 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.548442 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.548452 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.548835 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.549180 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.549185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.549189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.549339 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.549826 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.549834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.550005 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.550220 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.550699 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":269,"threads_started":1}
I20260812 06:52:01.551812 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.528805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.17.69.124:36451)
0812 06:52:01.533643 (+ 4838us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.533661 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.533707 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:52:01.535499 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.535521 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.535880 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.536213 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.537625 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.539236 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.540434 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.541029 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.541217 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.545672 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.545681 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.545691 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.545722 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.548361 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.548978 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.548984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.548986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.549132 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.549484 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.549488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.549490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.549923 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.550205 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.550554 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.550823 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":4587,"spinlock_wait_cycles":12032,"thread_start_us":4344,"threads_started":1}
I20260812 06:52:01.553300 30072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.124@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
I20260812 06:52:01.564759 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.553863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49104)
0812 06:52:01.554033 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.554050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.554143 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.554893 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.554896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.555196 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.555314 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.555322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.556535 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.556539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.557410 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.557421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.558004 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.558008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.558182 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.558685 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.558701 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.560501 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.563033 (+ 2532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.563037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.563040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.563382 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.563673 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.563676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.563678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.563794 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.564166 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.564168 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.564271 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.564369 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.564552 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 06:52:01.564841 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.554016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.17.69.125:36565)
0812 06:52:01.554642 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.554646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.554662 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:52:01.554708 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.554712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.554767 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.554852 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.555460 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.556432 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.557512 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.557908 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.557992 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.560666 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.560671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.560674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.560703 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.562931 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.563467 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.563470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.563472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.563590 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.563865 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.563868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.563870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.564088 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.564198 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.564435 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.564577 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":378,"threads_started":1}
I20260812 06:52:01.565816 30072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.124@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
I20260812 06:52:01.576924 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.566330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52446)
0812 06:52:01.566493 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.566507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.566603 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.566857 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.566861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.567177 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.567299 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.567307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.568191 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.568195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.568996 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.569007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.569636 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.569641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.569806 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.570253 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.570276 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.572064 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.574870 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.574875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.574880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.575210 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.575568 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.575572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.575574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.575728 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.576217 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.576220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.576345 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.576448 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.576641 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 06:52:01.577036 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.566503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.17.69.124:36451)
0812 06:52:01.566636 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.566641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.566668 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:52:01.566706 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.566710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.566759 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.566827 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.567426 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.568077 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.569123 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.569501 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.569607 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.572231 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.572236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.572239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.572265 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.574753 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.575312 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.575317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.575320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.575451 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.575848 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.575852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.575855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.576116 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.576236 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.576512 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.576766 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":43}
I20260812 06:52:01.581097 30072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Bootstrap starting.
I20260812 06:52:01.584028 30072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:01.584940 30072 log.cc:824] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:01.587267 30072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: No bootstrap required, opened a new log
I20260812 06:52:01.593959 30072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } }
I20260812 06:52:01.594228 30072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:01.594305 30072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382f5faf48224ee1b4880786d7532819, State: Initialized, Role: FOLLOWER
I20260812 06:52:01.594803 30072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } }
I20260812 06:52:01.595527 30088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } } }
I20260812 06:52:01.595824 30088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:01.596397 30072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:52:01.604501 30111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:52:01.604698 30111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:52:01.605906 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 28862
I20260812 06:52:01.609999 29785 master_runner.cc:428] Detected that this master 382f5faf48224ee1b4880786d7532819 is joining an existing cluster
I20260812 06:52:01.610113 29785 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.69.124:36451
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.65@KRBTEST.COM
I20260812 06:52:01.625563 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.606946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.17.69.65:46729)
0812 06:52:01.607597 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.607603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.607626 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:52:01.607685 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.607690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.607849 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.608037 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.609358 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.610633 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.612044 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.612739 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.612857 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.617551 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.617567 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.617581 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.617630 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.621768 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.622694 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.622704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.622708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.622884 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.623389 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.623395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.623399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.623865 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.624142 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.624845 (+ 703us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.625127 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":333,"threads_started":1}
I20260812 06:52:01.627408 29091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:01.633105 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.617423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.17.69.126:45281)
0812 06:52:01.617602 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.617608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.618500 (+ 892us) server_negotiation.cc:407] Connection header received
0812 06:52:01.618770 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.618775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.618854 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.618927 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.620396 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.621477 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.623501 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.624143 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.624239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.626596 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.626603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.626609 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.626647 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.629951 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.630755 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.630761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.630764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.630921 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.631388 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.631393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.631396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.631755 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.631887 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.632509 (+ 622us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.632758 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 06:52:01.634128 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.617613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35858)
0812 06:52:01.618400 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.618419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.618604 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.619141 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.619147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.619827 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.620213 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.620228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.621661 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.621667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.623335 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.623350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.624512 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.624519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.624684 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.625308 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.625331 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.626393 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.630119 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.630124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.630128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.630600 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.631067 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.631071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.631075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.631256 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.631968 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.631972 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.632174 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.633377 (+ 1203us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.633681 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":408,"threads_started":1}
I20260812 06:52:01.659871 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.639630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52460)
0812 06:52:01.640715 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.640733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.640968 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.641396 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.641401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.642081 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.642716 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.642729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.645143 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.645148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.646850 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.646868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.648149 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.648155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.648301 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.652037 (+ 3736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.652060 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.653219 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.657077 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.657081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.657085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.657535 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.657957 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.657961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.657964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.658116 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.658817 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.658823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.658993 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.659147 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.659444 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920}
I20260812 06:52:01.660889 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.639353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.17.69.124:36451)
0812 06:52:01.639522 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.639529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.640883 (+ 1354us) server_negotiation.cc:407] Connection header received
0812 06:52:01.641099 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.641104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.641166 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.641227 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.643233 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.644921 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.646999 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.647783 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.647874 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.653420 (+ 5546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.653426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.653431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.653468 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.656943 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.657658 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.657662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.657665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.657828 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.658241 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.658246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.658249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.658663 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.658822 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.660340 (+ 1518us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.660553 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":65}
I20260812 06:52:01.671404 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.618710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.17.69.125:36565)
0812 06:52:01.619782 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.619789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.634485 (+ 14696us) server_negotiation.cc:407] Connection header received
0812 06:52:01.634726 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.634732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.634826 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.634895 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.636059 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.637147 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.639603 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.640320 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.640414 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.663736 (+ 23322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.663744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.663749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.663793 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.667284 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.669009 (+ 1725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.669018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.669021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.669186 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.669655 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.669660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.669664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.670029 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.670165 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.670805 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.671057 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920}
I20260812 06:52:01.672279 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.618882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49120)
0812 06:52:01.634330 (+ 15448us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.634377 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.634574 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.635099 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.635104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.635657 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.635885 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.635897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.637340 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.637345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.639432 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.639449 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.642476 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.642482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.642953 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.651924 (+ 8971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.651947 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.661627 (+ 9680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.667470 (+ 5843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.667476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.667480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.668020 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.669347 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.669351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.669355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.669512 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.670271 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.670276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.670461 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.671627 (+ 1166us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.671908 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15350,"thread_start_us":19811,"threads_started":1}
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.126@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
I20260812 06:52:01.677211 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.659966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49122)
0812 06:52:01.668369 (+ 8403us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.668386 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.668560 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.669732 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.669737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.669765 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:01.670165 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.670179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.671902 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.671907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.674387 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.674403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.676340 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.676347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.676530 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.676595 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.676687 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.676821 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8279,"thread_start_us":8131,"threads_started":1}
I20260812 06:52:01.678081 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.659380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.17.69.125:36565)
0812 06:52:01.669091 (+ 9711us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.669098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.669124 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:52:01.669234 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.669239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.669323 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.669462 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:01.670362 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.671741 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.674562 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.675803 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.675933 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.676178 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.677553 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.677691 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9524,"thread_start_us":9383,"threads_started":1}
I20260812 06:52:01.679126 29867 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 06:52:01.679868 29914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc9679c7494c476ca100b4489a0c9886. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } }
I20260812 06:52:01.680100 29914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: LEARNER
I20260812 06:52:01.682953 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.646047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52470)
0812 06:52:01.655589 (+ 9542us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.655636 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.655879 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.656679 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.656684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.657220 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.657669 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.657685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.659729 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.659736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.663495 (+ 3759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.663511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.669174 (+ 5663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.669180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.669353 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.672534 (+ 3181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.672557 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.676617 (+ 4060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.680515 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.680519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.680523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.681007 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.681380 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.681385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.681388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.681511 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.682029 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.682034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.682153 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.682259 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.682538 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9426,"thread_start_us":9180,"threads_started":1}
I20260812 06:52:01.682978 30130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.646534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.17.69.124:36451)
0812 06:52:01.651078 (+ 4544us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.651084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.655836 (+ 4752us) server_negotiation.cc:407] Connection header received
0812 06:52:01.656034 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.656039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.656118 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.656278 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.657874 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.659532 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.668170 (+ 8638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.668867 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.668983 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.676853 (+ 7870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.676862 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.676866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.676908 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.680356 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.681143 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.681147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.681149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.681271 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.681623 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.681627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.681629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.681901 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.682031 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.682409 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.682596 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":374,"server-negotiator.queue_time_us":4326,"thread_start_us":4179,"threads_started":1}
I20260812 06:52:01.684182 28745 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.69.124:36451: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "382f5faf48224ee1b4880786d7532819"
member_type: NON_VOTER
last_known_addr {
host: "127.17.69.124"
port: 36451
}
attrs {
promote: true
}
}
dest_uuid: "cc9679c7494c476ca100b4489a0c9886"
cas_config_opid_index: 5
I20260812 06:52:01.684975 28745 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } }
I20260812 06:52:01.686653 30126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 382f5faf48224ee1b4880786d7532819 (127.17.69.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.689296 30026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:01.690418 30126 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.69.124:36451
I20260812 06:52:01.693250 30026 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 06:52:01.694519 29785 master.cc:537] Master@127.17.69.124:36451 shutting down...
I20260812 06:52:01.695976 29867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 06:52:01.704922 29785 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:01.704807 30133 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:01.705286 29785 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:01.705467 29785 tablet_replica.cc:330] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: stopping tablet replica
I20260812 06:52:01.705369 30126 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 06:52:01.705761 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.706226 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:01.715895 29785 master.cc:552] Master@127.17.69.124:36451 shutdown complete.
I20260812 06:52:01.715972 29785 master_runner.cc:305] Clearing existing system tablet
I20260812 06:52:01.716343 29785 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:01.716426 29785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:01.726686 29867 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 382f5faf48224ee1b4880786d7532819 (127.17.69.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
W20260812 06:52:01.727545 28726 consensus_peers.cc:487] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 -> Peer 382f5faf48224ee1b4880786d7532819 (127.17.69.124:36451): Couldn't send request to peer 382f5faf48224ee1b4880786d7532819. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:52:01.728981 29785 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.006s sys 0.000s
I20260812 06:52:01.731773 29914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: NON_VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: true } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.732057 29914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: LEARNER
I20260812 06:52:01.732772 30138 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: attempting to promote NON_VOTER 283f83cd6ce041b49d02336f4773bf4a to VOTER
I20260812 06:52:01.732934 30146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:01.733345 29785 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:01.733779 29785 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:52:01.733654 30138 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } }
I20260812 06:52:01.733956 29785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "382f5faf48224ee1b4880786d7532819"
format_stamp: "Formatted at 2026-08-12 06:52:01 on dist-test-slave-6zbq"
server_key: "6b2c5c473355d90b10c34974ca4c84f0"
server_key_iv: "3dc07f7e90e73d59fd25c31a20b53aea"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.736527 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.735496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52482)
0812 06:52:01.736246 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.736360 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619}
I20260812 06:52:01.737346 29867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 LEARNER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 06:52:01.737951 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.737400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52484)
0812 06:52:01.737665 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.737744 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 06:52:01.737939 30136 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 06:52:01.738229 28726 consensus_peers.cc:487] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 -> Peer 382f5faf48224ee1b4880786d7532819 (127.17.69.124:36451): Couldn't send request to peer 382f5faf48224ee1b4880786d7532819. Status: Network error: Client connection negotiation failed: client connection to 127.17.69.124:36451: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:52:01.740348 30136 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 283f83cd6ce041b49d02336f4773bf4a (127.17.69.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.741050 29785 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:01.744601 29867 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 283f83cd6ce041b49d02336f4773bf4a (127.17.69.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.747910 29785 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 06:52:01.748116 29785 log.cc:1196] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 06:52:01.748575 30126 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.748883 30126 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:01.748822 29785 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Deleting consensus metadata
I20260812 06:52:01.749171 29785 master_runner.cc:315] Copying system tablet from 127.17.69.126:45281
I20260812 06:52:01.749141 30126 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.749378 30126 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:01.749819 29914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.750057 29914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:01.755645 29785 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: tablet copy: Beginning tablet copy session from remote peer at address 127.17.69.126:45281
I20260812 06:52:01.772693 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.756371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.17.69.126:45281)
0812 06:52:01.756573 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:01.756579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:01.759003 (+ 2424us) server_negotiation.cc:407] Connection header received
0812 06:52:01.759188 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:01.759194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:01.759287 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:01.759389 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:01.760415 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.761553 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.763363 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.764007 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.764130 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.766366 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:01.766373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:01.766378 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:01.766418 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:01.769684 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.770479 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.770485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.770489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.770643 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:01.771084 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:01.771090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:01.771094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:01.771431 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:01.771593 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:01.772054 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 06:52:01.772318 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 06:52:01.772781 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:01.756335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35870)
0812 06:52:01.758893 (+ 2558us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:01.758916 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:01.759055 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:01.759436 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:01.759441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:01.759979 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:01.760259 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.760271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.761717 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.761722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.763203 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:01.763219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:01.764161 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:01.764167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:01.764362 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:01.765081 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:01.765106 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:01.766139 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:01.769823 (+ 3684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.769828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.769832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.770308 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.770774 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:01.770780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:01.770783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:01.770950 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:01.771555 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:01.771561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:01.771757 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 06:52:01.771916 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 06:52:01.772229 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2407,"thread_start_us":2199,"threads_started":1}
I20260812 06:52:01.773666 28772 tablet_copy_service.cc:140] P cc9679c7494c476ca100b4489a0c9886: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 382f5faf48224ee1b4880786d7532819 ({username='oryx', principal='oryx/127.17.69.124@KRBTEST.COM'} at 127.0.0.1:35870)
I20260812 06:52:01.773842 28772 tablet_copy_service.cc:161] P cc9679c7494c476ca100b4489a0c9886: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 382f5faf48224ee1b4880786d7532819 at {username='oryx', principal='oryx/127.17.69.124@KRBTEST.COM'} at 127.0.0.1:35870: session id = 382f5faf48224ee1b4880786d7532819-00000000000000000000000000000000
I20260812 06:52:01.777390 28772 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 06:52:01.779312 29785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 06:52:01.786145 29785 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: tablet copy: Starting download of 0 data blocks...
I20260812 06:52:01.786432 29785 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: tablet copy: Starting download of 1 WAL segments...
I20260812 06:52:01.787284 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 28862
I20260812 06:52:01.794678 29785 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:52:01.799508 28772 tablet_copy_service.cc:342] P cc9679c7494c476ca100b4489a0c9886: Request end of tablet copy session 382f5faf48224ee1b4880786d7532819-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.69.124@KRBTEST.COM'} at 127.0.0.1:35870
I20260812 06:52:01.799659 28772 tablet_copy_service.cc:433] P cc9679c7494c476ca100b4489a0c9886: ending tablet copy session 382f5faf48224ee1b4880786d7532819-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 382f5faf48224ee1b4880786d7532819
I20260812 06:52:01.812239 29785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:52:01.813973 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:46729
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:45281
--webserver_port=38167
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:01.815169 30164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:01.819474 30165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:01.827222 29785 server_base.cc:1034] running on GCE node
W20260812 06:52:01.827634 30173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:01.828027 29785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:01.828581 29785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:01.829753 29785 hybrid_clock.cc:648] HybridClock initialized: now 1786517521829715 us; error 32 us; skew 500 ppm
I20260812 06:52:01.829964 29785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:01 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517521, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:01.831662 29785 init.cc:377] Logged in from keytab as oryx/127.17.69.124@KRBTEST.COM (short username oryx)
I20260812 06:52:01.832945 29785 webserver.cc:460] Webserver started at http://127.17.69.124:43655/ using document root <none> and password file <none>
I20260812 06:52:01.833266 29785 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:01.833360 29785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:01.835896 29785 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:52:01.837416 30191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:01.837780 29785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 06:52:01.837899 29785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "382f5faf48224ee1b4880786d7532819"
format_stamp: "Formatted at 2026-08-12 06:52:01 on dist-test-slave-6zbq"
server_key: "6b2c5c473355d90b10c34974ca4c84f0"
server_key_iv: "3dc07f7e90e73d59fd25c31a20b53aea"
server_key_version: "encryptionkey@0"
I20260812 06:52:01.838116 29785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:01.869522 29785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:01.873198 29785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:01.919360 29785 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.124:36451
I20260812 06:52:01.919417 30290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.124:36451 every 8 connection(s)
I20260812 06:52:01.920327 29785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
I20260812 06:52:01.925824 30292 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:52:01.930259 30292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Bootstrap starting.
I20260812 06:52:01.945392 30292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:52:01.945811 30292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: Bootstrap complete.
I20260812 06:52:01.946682 30292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } }
I20260812 06:52:01.946920 30292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 382f5faf48224ee1b4880786d7532819, State: Initialized, Role: LEARNER
I20260812 06:52:01.947139 30292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } }
I20260812 06:52:01.947883 30296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:01.948168 30292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:01.948339 30296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:52:01.956036 30325 catalog_manager.cc:1259] Loaded cluster ID: db88e73cc6b743f68c1fe768ac5d5bc2
I20260812 06:52:01.956117 30325 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: loading cluster ID for follower catalog manager: success
I20260812 06:52:01.958565 30325 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: acquiring CA information for follower catalog manager: success
I20260812 06:52:01.960292 30325 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:52:02.201109 30178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:02.201361 30178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:02.201491 30178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:02.208673 30178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:02.208770 30178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:02.208902 30178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:02.218231 30178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:46729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=38167
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:02.219314 30178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:02.220446 30178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:02.233718 30332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:02.233743 30334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:02.233670 30336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:02.234652 30178 server_base.cc:1034] running on GCE node
I20260812 06:52:02.235239 30178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:02.236037 30178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:02.237233 30178 hybrid_clock.cc:648] HybridClock initialized: now 1786517522237172 us; error 51 us; skew 500 ppm
I20260812 06:52:02.237504 30178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:02.240603 30178 init.cc:377] Logged in from keytab as oryx/127.17.69.65@KRBTEST.COM (short username oryx)
I20260812 06:52:02.241861 30178 webserver.cc:460] Webserver started at http://127.17.69.65:38167/ using document root <none> and password file <none>
I20260812 06:52:02.242542 30178 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:02.242671 30178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:02.247512 30178 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:52:02.250310 30343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.251590 30178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:02.251715 30178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "973e161db5644dd2b5c260e6129d7f2a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "09e9451791af2bb830d5c92120bd72a8"
server_key_iv: "90953853067ea3ddef84815a9829c2ae"
server_key_version: "encryptionkey@0"
I20260812 06:52:02.252599 30178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:02.289515 30178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:02.293715 30178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:02.293954 30178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:02.294900 30178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:02.296239 30178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:02.296357 30178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.296465 30178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:02.296542 30178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.298012 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.291908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52486)
0812 06:52:02.292345 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.292365 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.292499 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.292954 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.292959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.292989 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:02.293165 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.293175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.294491 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.294496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.295878 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.295897 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.297444 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.297449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.297585 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.297612 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:52:02.297689 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:52:02.297797 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":216,"threads_started":1}
I20260812 06:52:02.298228 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.292084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52486 (local address 127.17.69.124:36451)
0812 06:52:02.292657 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.292662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.292677 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:52:02.292736 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.292740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.292807 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.292906 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:02.293317 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.294327 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.296062 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.297280 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.297435 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.297630 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:02.297720 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 06:52:02.297850 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":312,"threads_started":1}
I20260812 06:52:02.298982 30237 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 06:52:02.299657 30296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc9679c7494c476ca100b4489a0c9886. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:02.299913 30296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: LEARNER
I20260812 06:52:02.359078 30178 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:46729
I20260812 06:52:02.359494 30553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:46729 every 8 connection(s)
I20260812 06:52:02.360932 30178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:02.368595 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 30178
I20260812 06:52:02.369122 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 29117
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.66@KRBTEST.COM
I20260812 06:52:02.402426 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.365326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.17.69.65:55145)
0812 06:52:02.370108 (+ 4782us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.370508 (+ 400us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.373918 (+ 3410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.374988 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.375002 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.376283 (+ 1281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.377599 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.377652 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.379263 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.379269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.382371 (+ 3102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.382387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.384158 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.384167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.384607 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.385453 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.385487 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.389571 (+ 4084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:02.396163 (+ 6592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.396172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.396195 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.396895 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.397394 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.397399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.397403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.397712 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.398866 (+ 1154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:02.398874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:02.399415 (+ 541us) client_negotiation.cc:769] Sending connection context
0812 06:52:02.399752 (+ 337us) client_negotiation.cc:240] Negotiation successful
0812 06:52:02.400429 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4302,"mutex_wait_us":5,"thread_start_us":1399,"threads_started":1}
I20260812 06:52:02.403146 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.368257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:40317 (local address 127.17.69.126:45281)
0812 06:52:02.369251 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.369256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.378572 (+ 9316us) server_negotiation.cc:407] Connection header received
0812 06:52:02.378917 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.378924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.379008 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.379092 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.381816 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.383325 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.386451 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.387155 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.387261 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.392806 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.392814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.392819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.392862 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:02.398481 (+ 5619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.399988 (+ 1507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.399993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.399997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.400263 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.400731 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.400736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.400739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.401194 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:02.401329 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:02.402025 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 06:52:02.402250 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":609,"threads_started":1}
I20260812 06:52:02.403857 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.364969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:55145 (local address 127.17.69.124:36451)
0812 06:52:02.365209 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.365216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.371080 (+ 5864us) server_negotiation.cc:407] Connection header received
0812 06:52:02.374196 (+ 3116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.374204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.374300 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.374434 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.377832 (+ 3398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.379034 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.382545 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.383427 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.383601 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.389913 (+ 6312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.389920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.389925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.389967 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:02.396049 (+ 6082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.397043 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.397049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.397054 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.397249 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.397859 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.397864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.397867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.398402 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:02.402810 (+ 4408us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:02.402962 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 06:52:02.403269 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:52:02.404523 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.368096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:40317)
0812 06:52:02.378473 (+ 10377us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.378491 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.378719 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.379415 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.379419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.380669 (+ 1250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.381585 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.381600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.383512 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.383516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.386250 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.386265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.387698 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.387715 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.388015 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.388730 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.388757 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.392520 (+ 3763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:02.398622 (+ 6102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.398627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.398630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.399848 (+ 1218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.400427 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.400431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.400435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.400593 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.401415 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:02.401419 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:02.401653 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 06:52:02.403419 (+ 1766us) client_negotiation.cc:240] Negotiation successful
0812 06:52:02.403716 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8105,"mutex_wait_us":1918,"thread_start_us":5824,"threads_started":1}
I20260812 06:52:02.405782 30554 heartbeater.cc:347] Connected to a master server at 127.17.69.124:36451
I20260812 06:52:02.406031 30558 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:52:02.406217 30558 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:02.406268 30554 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:02.407441 30554 heartbeater.cc:511] Master 127.17.69.124:36451 requested a full tablet report, sending...
I20260812 06:52:02.407894 30558 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:52:02.408145 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.376551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:36539 (local address 127.17.69.125:36565)
0812 06:52:02.377146 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.377152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.379001 (+ 1849us) server_negotiation.cc:407] Connection header received
0812 06:52:02.379076 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.379082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.381576 (+ 2494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.381695 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.384069 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.385633 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.388460 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.389245 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.389355 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.394873 (+ 5518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.394882 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.394887 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.394925 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:02.400882 (+ 5957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.401824 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.401829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.401833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.401995 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.402551 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.402557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.402561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.402924 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:02.403068 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:02.407413 (+ 4345us) server_negotiation.cc:299] Negotiation successful
0812 06:52:02.407619 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":376,"threads_started":1}
I20260812 06:52:02.409162 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.376212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.17.69.65:36539)
0812 06:52:02.378707 (+ 2495us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.378727 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.378889 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.381975 (+ 3086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.381981 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.383092 (+ 1111us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.383907 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.383922 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.385814 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.385819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.388208 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.388224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.389761 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.389770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.389953 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.390707 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.390736 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.394560 (+ 3824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:02.401046 (+ 6486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.401051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.401055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.401677 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.402148 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:02.402153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:02.402157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:02.402407 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:02.403142 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:02.403147 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:02.403308 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 06:52:02.403427 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 06:52:02.408649 (+ 5222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2294,"mutex_wait_us":21,"thread_start_us":2001,"threads_started":1}
I20260812 06:52:02.409871 28745 ts_manager.cc:194] Re-registered known tserver with Master: 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729)
I20260812 06:52:02.410003 30559 heartbeater.cc:347] Connected to a master server at 127.17.69.125:36565
I20260812 06:52:02.410131 30559 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:02.410527 30559 heartbeater.cc:511] Master 127.17.69.125:36565 requested a full tablet report, sending...
I20260812 06:52:02.410519 30212 ts_manager.cc:194] Registered new tserver with Master: 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729)
I20260812 06:52:02.410961 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:40317
I20260812 06:52:02.412369 29846 ts_manager.cc:194] Registered new tserver with Master: 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729)
I20260812 06:52:02.413455 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.370696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33492 (local address 127.17.69.66:36363)
0812 06:52:02.383167 (+ 12471us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.383175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.383207 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:52:02.383276 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.383286 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.383565 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.387344 (+ 3779us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.387682 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.389856 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.392853 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.394054 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.401924 (+ 7870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.402283 (+ 359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.402298 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.402313 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.402385 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:02.408742 (+ 6357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.409772 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.409779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.409782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.410018 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:02.410526 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:02.410531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:02.410534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:02.410980 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:02.412508 (+ 1528us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:02.412641 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 06:52:02.412955 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12257,"thread_start_us":12955,"threads_started":1}
I20260812 06:52:02.414767 29339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:02.496366 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 29117
I20260812 06:52:02.520366 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:36363
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:45281
--webserver_port=42243
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:02.846446 30578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:02.846686 30578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:02.846813 30578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:02.853734 30578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:02.853840 30578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:02.853994 30578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:02.863314 30578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:36363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=42243
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:02.864445 30578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:02.865705 30578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:52:02.870918 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: NON_VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: true } } }
I20260812 06:52:02.871217 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:02.874047 30591 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: attempting to promote NON_VOTER 382f5faf48224ee1b4880786d7532819 to VOTER
I20260812 06:52:02.874667 30591 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } }
I20260812 06:52:02.877494 30237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 LEARNER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 06:52:02.877723 29867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Refusing update from remote peer cc9679c7494c476ca100b4489a0c9886: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 06:52:02.878154 30589 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:52:02.878548 30136 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 06:52:02.881323 30589 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 382f5faf48224ee1b4880786d7532819 (127.17.69.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
W20260812 06:52:02.881994 30597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:02.881913 29867 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 382f5faf48224ee1b4880786d7532819 (127.17.69.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.882802 30237 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 382f5faf48224ee1b4880786d7532819 (127.17.69.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.885313 30591 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.885371 30589 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.885602 30591 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:02.885682 30589 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:02.885808 30591 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.886062 30591 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:02.888759 30614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.889190 30614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 06:52:02.889657 30604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:02.891209 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:02.891331 30611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc9679c7494c476ca100b4489a0c9886" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:02.891664 30611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:02.893079 30578 server_base.cc:1034] running on GCE node
I20260812 06:52:02.893680 30578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:02.894596 30578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:02.895797 30578 hybrid_clock.cc:648] HybridClock initialized: now 1786517522895749 us; error 36 us; skew 500 ppm
I20260812 06:52:02.896086 30578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:02.899014 30578 init.cc:377] Logged in from keytab as oryx/127.17.69.66@KRBTEST.COM (short username oryx)
I20260812 06:52:02.900336 30578 webserver.cc:460] Webserver started at http://127.17.69.66:42243/ using document root <none> and password file <none>
I20260812 06:52:02.900964 30578 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:02.901104 30578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:02.906801 30578 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:52:02.909435 30633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.910729 30578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:52:02.910884 30578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "ef735495ae43443aa42cca77b43ea95a"
format_stamp: "Formatted at 2026-08-12 06:51:59 on dist-test-slave-6zbq"
server_key: "995178fd3a0db466ea83f75b2776d412"
server_key_iv: "84959b4cc51493f866b94da0c7b4436d"
server_key_version: "encryptionkey@0"
I20260812 06:52:02.911862 30578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:02.935798 30578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:02.939440 30578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:02.939682 30578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:02.940570 30578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:02.941883 30578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:02.941983 30578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.942086 30578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:02.942157 30578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:02.971679 30578 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:36363
I20260812 06:52:02.971743 30836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:36363 every 8 connection(s)
I20260812 06:52:02.973053 30578 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:02.976799 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 30578
I20260812 06:52:02.977195 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 29355
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517522, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
Aug 12 06:52:02 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.67@KRBTEST.COM
I20260812 06:52:03.008904 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.975723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:57055 (local address 127.17.69.126:45281)
0812 06:52:02.976256 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.976260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.977050 (+ 790us) server_negotiation.cc:407] Connection header received
0812 06:52:02.984474 (+ 7424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.984482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.984606 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.984756 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.987843 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.989093 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.992085 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.992821 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.992921 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.999390 (+ 6469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.999396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.999402 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.999490 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.004954 (+ 5464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.005947 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.005952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.005956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.006201 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.006840 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.006846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.006849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.007274 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.007424 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.008095 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.008376 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":297,"threads_started":1}
I20260812 06:52:03.009430 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.979050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37517 (local address 127.17.69.124:36451)
0812 06:52:02.982930 (+ 3880us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.982937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.982958 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:52:02.984658 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.984664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.984749 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.986050 (+ 1301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.989034 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.990896 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.992926 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.994196 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.994315 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.999432 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.999442 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.999447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.999486 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.004415 (+ 4929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.005584 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.005589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.005592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.005823 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.006394 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.006399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.006402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.006837 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.007305 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.007833 (+ 528us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.008387 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3724,"thread_start_us":3482,"threads_started":1}
I20260812 06:52:03.009603 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.977534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.17.69.66:37677)
0812 06:52:02.978481 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.978499 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.984181 (+ 5682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.986172 (+ 1991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.986176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.987061 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.987717 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.987733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.989304 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.989310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.991824 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.991839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.992879 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.992888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.993170 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.993928 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.993974 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.997988 (+ 4014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.004048 (+ 6060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.004056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.004069 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.005012 (+ 943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.005536 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.005540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.005543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.005928 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.006726 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.006734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.007163 (+ 429us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.007510 (+ 347us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.008022 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":140,"spinlock_wait_cycles":16768,"thread_start_us":402,"threads_started":1}
I20260812 06:52:03.010203 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.979441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.17.69.66:37517)
0812 06:52:02.980126 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.980144 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.984492 (+ 4348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.985162 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.985171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.987834 (+ 2663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.988819 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.988833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.991055 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.991060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.992766 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.992779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.994504 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.994512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.994766 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.995632 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.995655 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.999057 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.004573 (+ 5516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.004578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.004581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.005455 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.005999 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.006003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.006007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.006228 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.006998 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.007003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.007167 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.007510 (+ 343us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.008227 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":369,"thread_start_us":398,"threads_started":1}
I20260812 06:52:03.011044 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.977641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37677 (local address 127.17.69.125:36565)
0812 06:52:02.985812 (+ 8171us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.985818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.985838 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:52:02.985899 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.985904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.985979 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.986252 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.987896 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.989118 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.991991 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.992609 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.993580 (+ 971us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.998251 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:02.998258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:02.998263 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:02.998302 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.003885 (+ 5583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.005174 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.005179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.005183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.005401 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.006092 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.006097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.006100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.006560 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.010108 (+ 3548us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.010234 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.010467 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8052,"thread_start_us":7874,"threads_started":1}
I20260812 06:52:03.011531 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.976158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:57055)
0812 06:52:02.976905 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:02.976953 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:02.984181 (+ 7228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:02.985158 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:02.985171 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:02.986086 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:02.987609 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.987641 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.989334 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.989339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.991915 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:02.991931 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.993273 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.993280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:02.993524 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:02.994112 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:02.994139 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:02.999102 (+ 4963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.005108 (+ 6006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.005113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.005117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.005799 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.006399 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.006402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.006406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.006697 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.007513 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.007518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.007720 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.009374 (+ 1654us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.010541 (+ 1167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":119,"thread_start_us":304,"threads_started":1}
I20260812 06:52:03.011685 30837 heartbeater.cc:347] Connected to a master server at 127.17.69.124:36451
I20260812 06:52:03.012001 30842 heartbeater.cc:347] Connected to a master server at 127.17.69.125:36565
I20260812 06:52:03.012156 30837 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:03.013059 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:02.978661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.17.69.67:41185)
0812 06:52:02.979207 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:02.979215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:02.985101 (+ 5886us) server_negotiation.cc:407] Connection header received
0812 06:52:02.985301 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:02.985309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:02.985572 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:02.985731 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:02.988672 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.990267 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.992236 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:02.993170 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:02.993282 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.001866 (+ 8584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.001881 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.001897 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.001939 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.008136 (+ 6197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.009225 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.009234 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.009237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.009444 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.010438 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.010444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.010448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.010927 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.011314 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.012022 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.012323 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":560,"threads_started":1}
I20260812 06:52:03.013499 30837 heartbeater.cc:511] Master 127.17.69.124:36451 requested a full tablet report, sending...
I20260812 06:52:03.012189 30842 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:03.014890 30842 heartbeater.cc:511] Master 127.17.69.125:36565 requested a full tablet report, sending...
I20260812 06:52:03.015379 29587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:03.015780 30839 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:52:03.015964 30839 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:03.016202 30212 ts_manager.cc:194] Registered new tserver with Master: ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:52:03.016674 30839 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:52:03.017840 29846 ts_manager.cc:194] Registered new tserver with Master: ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:52:03.017880 28745 ts_manager.cc:194] Re-registered known tserver with Master: ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:52:03.019316 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:57055
I20260812 06:52:03.097781 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 29355
I20260812 06:52:03.120525 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:41185
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:45281
--webserver_port=42277
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:52:03.414085 30558 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
W20260812 06:52:03.439694 30862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:03.439970 30862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:03.440100 30862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:03.447110 30862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:03.447217 30862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:03.447359 30862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:03.456666 30862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:41185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=42277
--tserver_master_addrs=127.17.69.126:45281,127.17.69.125:36565,127.17.69.124:36451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:03.457852 30862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:03.459120 30862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:03.471414 30874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:03.471459 30870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:03.471475 30876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:03.472188 30862 server_base.cc:1034] running on GCE node
I20260812 06:52:03.472707 30862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:03.473533 30862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:03.474722 30862 hybrid_clock.cc:648] HybridClock initialized: now 1786517523474670 us; error 43 us; skew 500 ppm
I20260812 06:52:03.475009 30862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:03 dist-test-slave-6zbq krb5kdc[28674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517523, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:03.477962 30862 init.cc:377] Logged in from keytab as oryx/127.17.69.67@KRBTEST.COM (short username oryx)
I20260812 06:52:03.479270 30862 webserver.cc:460] Webserver started at http://127.17.69.67:42277/ using document root <none> and password file <none>
I20260812 06:52:03.479858 30862 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:03.479969 30862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:03.484836 30862 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:52:03.487250 30887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:03.488408 30862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:52:03.488545 30862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595"
format_stamp: "Formatted at 2026-08-12 06:52:00 on dist-test-slave-6zbq"
server_key: "02b692dd8547f8c2a38ebe5d0550e7d6"
server_key_iv: "ba644d32add0128f760a3ae826c52285"
server_key_version: "encryptionkey@0"
I20260812 06:52:03.489439 30862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:03.503057 30862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:03.507009 30862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:03.507270 30862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:03.508126 30862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:03.509461 30862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:03.509554 30862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:03.509655 30862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:03.509725 30862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:03.543780 30862 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41185
I20260812 06:52:03.543840 31091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41185 every 8 connection(s)
I20260812 06:52:03.545085 30862 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:03.552498 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 30862
Aug 12 06:52:03 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517523, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.125@KRBTEST.COM
I20260812 06:52:03.575878 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.547626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:50825 (local address 127.17.69.125:36565)
0812 06:52:03.548147 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.548154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.549240 (+ 1086us) server_negotiation.cc:407] Connection header received
0812 06:52:03.552105 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.552112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.552209 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.552292 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.554681 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.559119 (+ 4438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.562300 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.563011 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.563115 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.568192 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.568200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.568205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.568249 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.571837 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.572862 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.572869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.572873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.573114 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.573613 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.573619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.573622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.574118 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.574247 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.575263 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.575493 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":229,"threads_started":1}
I20260812 06:52:03.579066 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.548163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.17.69.67:50825)
0812 06:52:03.549055 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.549104 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.551756 (+ 2652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.552922 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.552932 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.553652 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:03.554473 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.554500 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.559507 (+ 5007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.559512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.561707 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.561734 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.563469 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.563477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.563825 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.564636 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:03.564666 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:03.567927 (+ 3261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.572038 (+ 4111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.572046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.572060 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.572682 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.573246 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.573251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.573255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.573484 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.574389 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.574397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.574842 (+ 445us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.576376 (+ 1534us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.577017 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":3,"thread_start_us":316,"threads_started":1}
I20260812 06:52:03.581094 31100 heartbeater.cc:347] Connected to a master server at 127.17.69.125:36565
I20260812 06:52:03.581624 31100 heartbeater.cc:464] Registering TS with master...
Aug 12 06:52:03 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517523, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:03.583216 31100 heartbeater.cc:511] Master 127.17.69.125:36565 requested a full tablet report, sending...
I20260812 06:52:03.585672 29846 ts_manager.cc:194] Registered new tserver with Master: 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
Aug 12 06:52:03 dist-test-slave-6zbq krb5kdc[28674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517523, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.124@KRBTEST.COM
I20260812 06:52:03.600193 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.549545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:37161 (local address 127.17.69.124:36451)
0812 06:52:03.562198 (+ 12653us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.562205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.569010 (+ 6805us) server_negotiation.cc:407] Connection header received
0812 06:52:03.569503 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.569509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.569606 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.569700 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.572797 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.573694 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.581679 (+ 7985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.582653 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.582763 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.590957 (+ 8194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.590965 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.590969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.591008 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.595918 (+ 4910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.597089 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.597094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.597097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.597381 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.597875 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.597881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.597884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.598286 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.598453 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.599621 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.599839 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12515,"thread_start_us":12407,"threads_started":1}
I20260812 06:52:03.600920 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.562879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:37377)
0812 06:52:03.569357 (+ 6478us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.569375 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.569468 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.570481 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.570486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.571123 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:03.571766 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.571778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.573459 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.573464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.576347 (+ 2883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.576363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.577740 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.577749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.578031 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.579385 (+ 1354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:03.579406 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:03.583905 (+ 4499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.589777 (+ 5872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.589784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.589788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.590948 (+ 1160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.597613 (+ 6665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.597618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.597622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.598177 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.599315 (+ 1138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.599321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.599473 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.599603 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.599952 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6362,"mutex_wait_us":26,"thread_start_us":6645,"threads_started":1}
I20260812 06:52:03.601461 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.563011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:37377 (local address 127.17.69.126:45281)
0812 06:52:03.569988 (+ 6977us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.569994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.570010 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:52:03.570068 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.570073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.570155 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.570274 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.571932 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.573278 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.576491 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.577429 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.577559 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.584125 (+ 6566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.584132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.584136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.584184 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.589613 (+ 5429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.591937 (+ 2324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.591942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.591945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.596039 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.598531 (+ 2492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.598536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.598539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.598874 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.599034 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.600622 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.600844 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6858,"thread_start_us":6793,"threads_started":1}
I20260812 06:52:03.602626 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.549330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.17.69.67:37161)
0812 06:52:03.562774 (+ 13444us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.568917 (+ 6143us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.569194 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.570001 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.570005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.570754 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:03.572615 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.572631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.573859 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.573864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.579004 (+ 5140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.579023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.584303 (+ 5280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.584312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.584573 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.585298 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:03.585323 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:03.590698 (+ 5375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:03.596214 (+ 5516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.596229 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.596233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.596947 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.597532 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:03.597537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:03.597540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:03.597737 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:03.599121 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:03.599126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:03.599311 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.601935 (+ 2624us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.602207 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13317,"mutex_wait_us":47,"thread_start_us":13164,"threads_started":1}
I20260812 06:52:03.605715 31093 heartbeater.cc:347] Connected to a master server at 127.17.69.124:36451
I20260812 06:52:03.605880 31093 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:03.606459 31093 heartbeater.cc:511] Master 127.17.69.124:36451 requested a full tablet report, sending...
I20260812 06:52:03.607513 31097 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45281
I20260812 06:52:03.607648 31097 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:03.608040 31097 heartbeater.cc:511] Master 127.17.69.126:45281 requested a full tablet report, sending...
I20260812 06:52:03.609534 28745 ts_manager.cc:194] Re-registered known tserver with Master: 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
I20260812 06:52:03.609774 30212 ts_manager.cc:194] Registered new tserver with Master: 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
I20260812 06:52:03.610770 28745 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:37377
I20260812 06:52:03.612514 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.580999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.17.69.125:36565)
0812 06:52:03.581267 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.581274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.583483 (+ 2209us) server_negotiation.cc:407] Connection header received
0812 06:52:03.583741 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.583746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.583819 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.583885 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.591622 (+ 7737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.593216 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.596061 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.597254 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.597352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.605389 (+ 8037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.605394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.605397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.605427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.608873 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.609656 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.609662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.609664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.609822 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.610461 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.610466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.610470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.610806 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.611089 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.611442 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.612222 (+ 780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 06:52:03.615651 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.573898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.17.69.126:45281)
0812 06:52:03.578927 (+ 5029us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.578931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.580331 (+ 1400us) server_negotiation.cc:407] Connection header received
0812 06:52:03.580501 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.580506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.580583 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.580739 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.591658 (+ 10919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.593091 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.595959 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.597035 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.597120 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.601505 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.601512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.601517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.601551 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.607071 (+ 5520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.611683 (+ 4612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.611688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.611691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.611934 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.612431 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.612436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.612439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.614170 (+ 1731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.614438 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.615004 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.615223 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4948,"thread_start_us":4794,"threads_started":1}
I20260812 06:52:03.619452 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.583188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.17.69.124:36451)
0812 06:52:03.600432 (+ 17244us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.600438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.601855 (+ 1417us) server_negotiation.cc:407] Connection header received
0812 06:52:03.603129 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.603134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.603239 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.603332 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:03.605692 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.606873 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.608756 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.609469 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.609634 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.613291 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:03.613297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:03.613301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:03.613337 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:03.616862 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.617577 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.617582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.617584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.617712 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:03.618213 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:03.618217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:03.618219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:03.618523 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:03.618766 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.619085 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.619241 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17143,"thread_start_us":20915,"threads_started":1}
I20260812 06:52:03.619797 28745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:03.629052 28745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:03.663332 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.649144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41185 (local address 127.0.0.1:57284)
0812 06:52:03.651289 (+ 2145us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.651305 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.651421 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.652568 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.652574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.652614 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:03.653030 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.653043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.655846 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.655852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.658880 (+ 3028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.658899 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.662245 (+ 3346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.662252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.662470 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.662510 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.662602 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.662774 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2025,"thread_start_us":1890,"threads_started":1}
I20260812 06:52:03.665125 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.649636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.17.69.67:41185)
0812 06:52:03.651160 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.651171 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.651545 (+ 374us) server_negotiation.cc:407] Connection header received
0812 06:52:03.651677 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.651685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.651957 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.652225 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:03.653307 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.655667 (+ 2360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.659216 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.661692 (+ 2476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.661844 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.662092 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.664275 (+ 2183us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.664526 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":1046,"threads_started":1}
I20260812 06:52:03.669986 30976 tablet_service.cc:1467] Processing CreateTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 (DEFAULT_TABLE table=test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:03.671607 30976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8099d96afd4e778bd613cf39bc7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:03.671725 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.651335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36363 (local address 127.0.0.1:33504)
0812 06:52:03.663967 (+ 12632us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.663988 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.664101 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.664728 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.664733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.664761 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:03.664938 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.664950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.666720 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.666725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.669051 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.669069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.670990 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.670997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.671178 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.671212 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.671275 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.671405 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12518,"thread_start_us":14865,"threads_started":1}
I20260812 06:52:03.672591 31174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.651616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.17.69.66:36363)
0812 06:52:03.664152 (+ 12536us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.664161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.664191 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:52:03.664279 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.664285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.664471 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.664776 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:03.665133 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.666551 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.669260 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.670805 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.671979 (+ 1174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.672206 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.672275 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.672393 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12387,"thread_start_us":12267,"threads_started":1}
I20260812 06:52:03.675611 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.648365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46729 (local address 127.0.0.1:32768)
0812 06:52:03.649194 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.649216 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.649431 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.666254 (+ 16823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.666259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.666286 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:03.666795 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.666808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.669846 (+ 3038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.669851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.672111 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.672125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.674882 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.674889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.675073 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.675107 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.675182 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.675303 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":419,"threads_started":1}
I20260812 06:52:03.676191 30719 tablet_service.cc:1467] Processing CreateTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 (DEFAULT_TABLE table=test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:03.677210 30719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8099d96afd4e778bd613cf39bc7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:03.677454 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.648814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.17.69.65:46729)
0812 06:52:03.665433 (+ 16619us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.665441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.665467 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:52:03.665563 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.665570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.665778 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.667640 (+ 1862us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:03.668019 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.669693 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.672279 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.674645 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.676413 (+ 1768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.676727 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.676802 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.676972 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16449,"thread_start_us":17146,"threads_started":1}
I20260812 06:52:03.681281 30437 tablet_service.cc:1467] Processing CreateTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 (DEFAULT_TABLE table=test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:03.682713 30437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8099d96afd4e778bd613cf39bc7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:03.691781 31176 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Bootstrap starting.
I20260812 06:52:03.692806 31177 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Bootstrap starting.
I20260812 06:52:03.696919 31177 tablet_bootstrap.cc:654] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:03.697171 31176 tablet_bootstrap.cc:654] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:03.698683 31178 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Bootstrap starting.
I20260812 06:52:03.698709 31177 log.cc:824] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:03.699115 31176 log.cc:824] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:03.702230 31176 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: No bootstrap required, opened a new log
I20260812 06:52:03.702495 31176 ts_tablet_manager.cc:1397] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260812 06:52:03.703086 31177 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: No bootstrap required, opened a new log
I20260812 06:52:03.703358 31177 ts_tablet_manager.cc:1397] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260812 06:52:03.703548 31178 tablet_bootstrap.cc:654] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:03.705296 31178 log.cc:824] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:03.708925 31178 tablet_bootstrap.cc:492] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: No bootstrap required, opened a new log
I20260812 06:52:03.709188 31178 ts_tablet_manager.cc:1397] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 06:52:03.714033 31176 raft_consensus.cc:348] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.714496 31176 raft_consensus.cc:374] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:03.714684 31176 raft_consensus.cc:729] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef735495ae43443aa42cca77b43ea95a, State: Initialized, Role: FOLLOWER
I20260812 06:52:03.715423 31176 consensus_queue.cc:260] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.716787 31177 raft_consensus.cc:348] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.717425 31177 raft_consensus.cc:374] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:03.717550 31177 raft_consensus.cc:729] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc3faf0ee1c4e459e3a6b3e0cd03595, State: Initialized, Role: FOLLOWER
I20260812 06:52:03.718139 31177 consensus_queue.cc:260] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.718525 31176 ts_tablet_manager.cc:1428] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 06:52:03.718824 30839 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
I20260812 06:52:03.724324 31178 raft_consensus.cc:348] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.724972 31178 raft_consensus.cc:374] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:03.725170 31178 raft_consensus.cc:729] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973e161db5644dd2b5c260e6129d7f2a, State: Initialized, Role: FOLLOWER
I20260812 06:52:03.725874 31178 consensus_queue.cc:260] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
W20260812 06:52:03.726547 30843 tablet.cc:2369] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:52:03.726672 31177 ts_tablet_manager.cc:1428] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Time spent starting tablet: real 0.023s user 0.014s sys 0.010s
I20260812 06:52:03.726991 31097 heartbeater.cc:503] Master 127.17.69.126:45281 was elected leader, sending a full tablet report...
I20260812 06:52:03.731598 31178 ts_tablet_manager.cc:1428] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Time spent starting tablet: real 0.022s user 0.011s sys 0.012s
W20260812 06:52:03.803371 31102 tablet.cc:2369] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:52:03.876840 30563 tablet.cc:2369] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:52:03.894915 31186 raft_consensus.cc:482] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:03.895241 31186 raft_consensus.cc:504] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.897168 31186 leader_election.cc:288] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729), ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:52:03.904352 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.897577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45839 (local address 127.17.69.65:46729)
0812 06:52:03.897850 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.897856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.897876 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:52:03.897947 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.897952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.898030 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.898114 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:03.898619 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.899640 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.901937 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.903265 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.903425 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.903502 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.903954 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.904095 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:52:03.904369 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.897377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46729 (local address 127.17.69.67:45839)
0812 06:52:03.897612 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.897631 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.897784 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.898215 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.898220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.898253 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:03.898476 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.898488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.899809 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.899815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.901774 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.901790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.903431 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.903437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.903764 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.903789 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.903909 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.904095 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":38}
I20260812 06:52:03.904659 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.897826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36363 (local address 127.17.69.67:58787)
0812 06:52:03.897999 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:03.898015 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:03.898095 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:03.898549 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:03.898553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:03.898582 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:03.898962 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.898977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.900573 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.900577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.902321 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:03.902333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.903884 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.903891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:03.904114 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.904139 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:52:03.904217 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:52:03.904346 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":27}
I20260812 06:52:03.905061 31174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.898021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:58787 (local address 127.17.69.66:36363)
0812 06:52:03.898236 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.898241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.898258 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:52:03.898327 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.898331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.898430 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.898528 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:03.899114 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.900390 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.902535 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.903720 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.903911 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.903986 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.904267 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.904394 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:52:03.905365 30473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8099d96afd4e778bd613cf39bc7dc6" candidate_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973e161db5644dd2b5c260e6129d7f2a" is_pre_election: true
I20260812 06:52:03.905967 30473 raft_consensus.cc:2454] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc3faf0ee1c4e459e3a6b3e0cd03595 in term 0.
I20260812 06:52:03.905958 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8099d96afd4e778bd613cf39bc7dc6" candidate_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef735495ae43443aa42cca77b43ea95a" is_pre_election: true
I20260812 06:52:03.906718 30754 raft_consensus.cc:2454] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc3faf0ee1c4e459e3a6b3e0cd03595 in term 0.
I20260812 06:52:03.906813 30895 leader_election.cc:302] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc3faf0ee1c4e459e3a6b3e0cd03595, 973e161db5644dd2b5c260e6129d7f2a; no voters:
I20260812 06:52:03.907227 31186 raft_consensus.cc:2781] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:03.907397 31186 raft_consensus.cc:482] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:03.907506 31186 raft_consensus.cc:3037] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:03.910140 31186 raft_consensus.cc:504] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.910725 31186 leader_election.cc:288] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [CANDIDATE]: Term 1 election: Requested vote from peers 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729), ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363)
I20260812 06:52:03.911284 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8099d96afd4e778bd613cf39bc7dc6" candidate_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef735495ae43443aa42cca77b43ea95a"
I20260812 06:52:03.911351 30473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8099d96afd4e778bd613cf39bc7dc6" candidate_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973e161db5644dd2b5c260e6129d7f2a"
I20260812 06:52:03.911489 30754 raft_consensus.cc:3037] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:03.911530 30473 raft_consensus.cc:3037] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:03.915107 30473 raft_consensus.cc:2454] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc3faf0ee1c4e459e3a6b3e0cd03595 in term 1.
I20260812 06:52:03.915135 30754 raft_consensus.cc:2454] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc3faf0ee1c4e459e3a6b3e0cd03595 in term 1.
I20260812 06:52:03.915586 30895 leader_election.cc:302] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc3faf0ee1c4e459e3a6b3e0cd03595, ef735495ae43443aa42cca77b43ea95a; no voters:
I20260812 06:52:03.915906 31186 raft_consensus.cc:2781] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:03.916587 31186 raft_consensus.cc:686] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc3faf0ee1c4e459e3a6b3e0cd03595, State: Running, Role: LEADER
I20260812 06:52:03.917263 31186 consensus_queue.cc:237] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } }
I20260812 06:52:03.923910 28745 catalog_manager.cc:5495] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } health_report { overall_health: UNKNOWN } } }
I20260812 06:52:03.972695 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:03.966873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.17.69.67:41185)
0812 06:52:03.967100 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:03.967105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:03.967122 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:52:03.967210 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:03.967214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:03.967291 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:03.967410 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:03.967728 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.968467 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.969998 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:03.970685 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:03.970780 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:03.971300 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:03.971950 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:03.972152 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:03.972217 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 06:52:03.972302 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:52:03.982658 30473 raft_consensus.cc:1261] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Refusing update from remote peer 7bc3faf0ee1c4e459e3a6b3e0cd03595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:03.982738 30754 raft_consensus.cc:1261] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Refusing update from remote peer 7bc3faf0ee1c4e459e3a6b3e0cd03595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:03.983614 31189 consensus_queue.cc:1035] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:03.984021 31186 consensus_queue.cc:1035] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:04.000154 31196 mvcc.cc:205] Tried to move back new op lower bound from 7317575778221940736 to 7317575777969606656. Current Snapshot: MvccSnapshot[applied={T|T < 7317575778221940736}]
I20260812 06:52:04.016829 28745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:04.017151 28745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:04.023057 28745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc9679c7494c476ca100b4489a0c9886: Sending DeleteTablet for 3 replicas of tablet de8099d96afd4e778bd613cf39bc7dc6
I20260812 06:52:04.024235 30437 tablet_service.cc:1514] Processing DeleteTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:04 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:32768
I20260812 06:52:04.025074 30976 tablet_service.cc:1514] Processing DeleteTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:04 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:57284
I20260812 06:52:04.025935 30719 tablet_service.cc:1514] Processing DeleteTablet for tablet de8099d96afd4e778bd613cf39bc7dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:04 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33504
I20260812 06:52:04.037341 31212 tablet_replica.cc:330] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: stopping tablet replica
I20260812 06:52:04.037792 31212 raft_consensus.cc:2229] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:04.038516 31212 raft_consensus.cc:2258] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:04.040845 31212 ts_tablet_manager.cc:1905] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:04.049530 31212 ts_tablet_manager.cc:1918] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:04.049706 31212 log.cc:1196] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/de8099d96afd4e778bd613cf39bc7dc6
I20260812 06:52:04.050328 31212 ts_tablet_manager.cc:1939] T de8099d96afd4e778bd613cf39bc7dc6 P 973e161db5644dd2b5c260e6129d7f2a: Deleting consensus metadata
I20260812 06:52:04.052242 28726 catalog_manager.cc:4841] TS 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729): tablet de8099d96afd4e778bd613cf39bc7dc6 (table test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]) successfully deleted
I20260812 06:52:04.055182 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 28695
I20260812 06:52:04.056504 28786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:04.056913 31216 tablet_replica.cc:330] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: stopping tablet replica
I20260812 06:52:04.057397 31216 raft_consensus.cc:2229] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:04.062763 31216 raft_consensus.cc:2258] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:04.064311 31215 tablet_replica.cc:330] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: stopping tablet replica
I20260812 06:52:04.065465 31216 ts_tablet_manager.cc:1905] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:04.064869 31215 raft_consensus.cc:2229] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:04.070947 31215 raft_consensus.cc:2258] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:04.072767 31215 ts_tablet_manager.cc:1905] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:04.073016 31216 ts_tablet_manager.cc:1918] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:04.073208 31216 log.cc:1196] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/de8099d96afd4e778bd613cf39bc7dc6
I20260812 06:52:04.073864 31216 ts_tablet_manager.cc:1939] T de8099d96afd4e778bd613cf39bc7dc6 P ef735495ae43443aa42cca77b43ea95a: Deleting consensus metadata
I20260812 06:52:04.080417 31215 ts_tablet_manager.cc:1918] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:04.080732 31215 log.cc:1196] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/de8099d96afd4e778bd613cf39bc7dc6
I20260812 06:52:04.081687 31215 ts_tablet_manager.cc:1939] T de8099d96afd4e778bd613cf39bc7dc6 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting consensus metadata
I20260812 06:52:04.089526 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:04.028720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.17.69.125:36565)
0812 06:52:04.028896 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:04.028903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:04.053545 (+ 24642us) server_negotiation.cc:407] Connection header received
0812 06:52:04.056868 (+ 3323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:04.056876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:04.056963 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:04.057076 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:04.062148 (+ 5072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.063668 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.067333 (+ 3665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.068177 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.068321 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:04.081607 (+ 13286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:04.081619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:04.081624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:04.081671 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:04.085869 (+ 4198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.086735 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.086740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.086742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.086866 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.087259 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.087264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.087267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.087727 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:04.088032 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:04.088718 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 06:52:04.088951 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:52:04.104615 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:04.054542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.17.69.124:36451)
0812 06:52:04.055257 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:04.055262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:04.089613 (+ 34351us) server_negotiation.cc:407] Connection header received
0812 06:52:04.090207 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:04.090214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:04.090300 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:04.090412 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:04.092714 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.093673 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.095220 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.095933 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.096052 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:04.098865 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:04.098871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:04.098874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:04.098926 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:04.101968 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.102703 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.102708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.102710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.102858 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.103205 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.103208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.103210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.103540 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:04.103767 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:04.104085 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 06:52:04.104294 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588}
I20260812 06:52:04.266722 28726 catalog_manager.cc:4841] TS 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185): tablet de8099d96afd4e778bd613cf39bc7dc6 (table test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]) successfully deleted
I20260812 06:52:04.267668 28726 catalog_manager.cc:4841] TS ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363): tablet de8099d96afd4e778bd613cf39bc7dc6 (table test-table [id=01c52ffb4b2e41129f9e7b940b1a6227]) successfully deleted
I20260812 06:52:04.268384 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 28695
I20260812 06:52:04.317690 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:04.301678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.17.69.125:36565)
0812 06:52:04.301895 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:04.301901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:04.301938 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:52:04.302050 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:04.302055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:04.302137 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:04.302237 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:04.303946 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.305060 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.307132 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.307852 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.307983 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:04.311375 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:04.311382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:04.311387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:04.311432 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:04.314545 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.315350 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.315355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.315359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.315544 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.315978 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.315983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.315986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.316434 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:04.316693 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:04.317046 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 06:52:04.317363 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:52:04.319397 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:04.302672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.17.69.124:36451)
0812 06:52:04.302850 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:04.302856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:04.304082 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 06:52:04.304225 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:04.304229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:04.304294 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:04.304375 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:04.306209 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.306948 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.308619 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:04.309276 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:04.309408 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:04.312578 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:04.312584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:04.312588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:04.312636 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:04.316083 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.316867 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.316873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.316877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.317040 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:04.317622 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:04.317628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:04.317631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:04.318274 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:04.318534 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:04.318878 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 06:52:04.319096 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:52:05.008525 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.007792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:42811)
0812 06:52:05.008213 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.008329 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":195,"threads_started":1}
W20260812 06:52:05.008963 31097 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:45281 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.011227 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.010545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:44895)
0812 06:52:05.010972 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.011083 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":210,"threads_started":1}
W20260812 06:52:05.011585 30839 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:45281 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.016391 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.015724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:37677)
0812 06:52:05.016127 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.016233 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":206,"threads_started":1}
W20260812 06:52:05.016709 30558 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:45281 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.524184 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc9679c7494c476ca100b4489a0c9886)
I20260812 06:52:05.524394 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } }
I20260812 06:52:05.525152 31239 leader_election.cc:288] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281), 382f5faf48224ee1b4880786d7532819 (127.17.69.124:36451)
I20260812 06:52:05.526460 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.525372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36294)
0812 06:52:05.525941 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.526166 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":245,"threads_started":1}
I20260812 06:52:05.526489 31242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc9679c7494c476ca100b4489a0c9886)
I20260812 06:52:05.526813 31242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } }
W20260812 06:52:05.527639 29819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.528482 31242 leader_election.cc:288] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281), 283f83cd6ce041b49d02336f4773bf4a (127.17.69.125:36565)
I20260812 06:52:05.529682 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.529174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36304)
0812 06:52:05.529313 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.529464 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 06:52:05.529851 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.528807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36300)
0812 06:52:05.529389 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.529594 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":336,"threads_started":1}
W20260812 06:52:05.530104 29819 leader_election.cc:334] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281): Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
W20260812 06:52:05.530767 30196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.532595 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.532055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36310)
0812 06:52:05.532262 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.532431 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:52:05.532747 30196 leader_election.cc:334] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281): Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.534296 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.525962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:36451 (local address 127.0.0.1:52530)
0812 06:52:05.526846 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.526865 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.527013 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.528168 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.528173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.528200 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.528478 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.528508 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.530120 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.530125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.532130 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.532147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.533617 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.533624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.533790 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.533814 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.533894 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.534024 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":1378,"threads_started":1}
I20260812 06:52:05.534413 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.526441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.17.69.124:36451)
0812 06:52:05.527880 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.527886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.527903 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:52:05.527965 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.527968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.528036 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.528126 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.528713 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.529927 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.532288 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.533440 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.533594 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.533703 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.533930 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.534069 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1321,"thread_start_us":1172,"threads_started":1}
I20260812 06:52:05.534966 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "283f83cd6ce041b49d02336f4773bf4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "382f5faf48224ee1b4880786d7532819" is_pre_election: true
I20260812 06:52:05.535245 30237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283f83cd6ce041b49d02336f4773bf4a in term 1.
I20260812 06:52:05.535698 29819 leader_election.cc:302] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 283f83cd6ce041b49d02336f4773bf4a, 382f5faf48224ee1b4880786d7532819; no voters: cc9679c7494c476ca100b4489a0c9886
I20260812 06:52:05.535979 31239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:52:05.536094 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc9679c7494c476ca100b4489a0c9886)
I20260812 06:52:05.536191 31239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:52:05.536718 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.529502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36565 (local address 127.0.0.1:49178)
0812 06:52:05.530036 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.530051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.530160 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.531068 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.531073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.531112 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.531494 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.531506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.532957 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.532962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.534373 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.534389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.536177 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.536183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.536362 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.536386 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.536456 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.536540 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":1013,"threads_started":1}
I20260812 06:52:05.536947 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.529803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.17.69.125:36565)
0812 06:52:05.530721 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.530727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.530748 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:52:05.530806 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.530812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.530875 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.531025 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.531688 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.532813 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.534858 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.535825 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.535939 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.536047 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.536503 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.536627 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":592,"threads_started":1}
I20260812 06:52:05.537524 29867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "382f5faf48224ee1b4880786d7532819" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "283f83cd6ce041b49d02336f4773bf4a" is_pre_election: true
I20260812 06:52:05.538851 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } }
I20260812 06:52:05.539439 29867 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 382f5faf48224ee1b4880786d7532819 in current term 2: Already voted for candidate 283f83cd6ce041b49d02336f4773bf4a in this term.
I20260812 06:52:05.539541 31239 leader_election.cc:288] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 election: Requested vote from peers cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281), 382f5faf48224ee1b4880786d7532819 (127.17.69.124:36451)
I20260812 06:52:05.540369 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.539957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36322)
0812 06:52:05.540110 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.540197 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 06:52:05.540443 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "283f83cd6ce041b49d02336f4773bf4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "382f5faf48224ee1b4880786d7532819"
I20260812 06:52:05.540657 30237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:52:05.541253 30196 leader_election.cc:302] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 382f5faf48224ee1b4880786d7532819; no voters: 283f83cd6ce041b49d02336f4773bf4a, cc9679c7494c476ca100b4489a0c9886
I20260812 06:52:05.541538 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.541101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36332)
0812 06:52:05.541317 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.541401 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260812 06:52:05.541699 29819 leader_election.cc:334] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281): Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
I20260812 06:52:05.543090 30237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283f83cd6ce041b49d02336f4773bf4a in term 2.
I20260812 06:52:05.543263 31242 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:52:05.543632 29819 leader_election.cc:302] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 283f83cd6ce041b49d02336f4773bf4a, 382f5faf48224ee1b4880786d7532819; no voters: cc9679c7494c476ca100b4489a0c9886
I20260812 06:52:05.543919 31239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:52:05.544358 31239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [term 2 LEADER]: Becoming Leader. State: Replica: 283f83cd6ce041b49d02336f4773bf4a, State: Running, Role: LEADER
I20260812 06:52:05.544739 31239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } }
I20260812 06:52:05.546737 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283f83cd6ce041b49d02336f4773bf4a. Latest consensus state: current_term: 2 leader_uuid: "283f83cd6ce041b49d02336f4773bf4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:05.547112 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:05.547726 31257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:05.551803 31257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:05.552585 31257 catalog_manager.cc:1259] Loaded cluster ID: db88e73cc6b743f68c1fe768ac5d5bc2
I20260812 06:52:05.552667 31257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:05.553758 31257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:05.554584 31257 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Loaded TSK: 0
I20260812 06:52:05.555423 31257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:52:05.597476 29841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:05.604290 29841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:05.608044 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.607646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36356)
0812 06:52:05.607785 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.607878 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 06:52:05.608351 30237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [term 2 FOLLOWER]: Refusing update from remote peer 283f83cd6ce041b49d02336f4773bf4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 06:52:05.608958 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.608586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:36360)
0812 06:52:05.608737 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.608823 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:52:05.608963 31255 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 06:52:05.609333 29819 consensus_peers.cc:487] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a -> Peer cc9679c7494c476ca100b4489a0c9886 (127.17.69.126:45281): Couldn't send request to peer cc9679c7494c476ca100b4489a0c9886. Status: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:52:05.611661 31242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283f83cd6ce041b49d02336f4773bf4a. Latest consensus state: current_term: 2 leader_uuid: "283f83cd6ce041b49d02336f4773bf4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:05.611896 31242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:05.613164 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "283f83cd6ce041b49d02336f4773bf4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:05.613499 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:05.613935 31242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "283f83cd6ce041b49d02336f4773bf4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc9679c7494c476ca100b4489a0c9886" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45281 } } peers { permanent_uuid: "283f83cd6ce041b49d02336f4773bf4a" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36565 } attrs { promote: false } } peers { permanent_uuid: "382f5faf48224ee1b4880786d7532819" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 36451 } attrs { promote: false } } }
I20260812 06:52:05.614212 31242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382f5faf48224ee1b4880786d7532819 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:52:05.643143 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.621354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36363 (local address 127.0.0.1:33510)
0812 06:52:05.621529 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.621552 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.621675 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.637164 (+ 15489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.637170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.637201 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.637394 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.637406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.638970 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.638976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.640923 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.640937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.642426 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.642433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.642603 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.642638 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.642706 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.642839 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":15}
I20260812 06:52:05.643594 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.621940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46729 (local address 127.0.0.1:32782)
0812 06:52:05.622648 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.622667 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.622780 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.638527 (+ 15747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.638532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.638558 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.638885 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.638896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.640056 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.640062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.641417 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.641429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.642903 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.642909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.643055 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.643082 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.643173 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.643315 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":4}
I20260812 06:52:05.643622 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.622075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.17.69.65:46729)
0812 06:52:05.638187 (+ 16112us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.638191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.638207 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:52:05.638270 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.638275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.638364 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.639010 (+ 646us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.639146 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.639936 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.641573 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.642635 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.642808 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.642912 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.643200 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.643338 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15998,"thread_start_us":15906,"threads_started":1}
I20260812 06:52:05.644029 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.621481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.17.69.66:36363)
0812 06:52:05.636710 (+ 15229us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.636716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.636735 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:52:05.636806 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.636811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.636893 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.637549 (+ 656us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.637717 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.638823 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.641098 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.642223 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.643447 (+ 1224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.643546 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.643595 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.643719 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15106,"thread_start_us":15487,"threads_started":1}
I20260812 06:52:05.644749 30719 tablet_service.cc:1467] Processing CreateTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 (DEFAULT_TABLE table=test-table [id=1a4327d66cc141fab408dac6951d387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:05.645184 30719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a05f2fb56f0491cb87934d54fc3d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:05.645159 30437 tablet_service.cc:1467] Processing CreateTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 (DEFAULT_TABLE table=test-table [id=1a4327d66cc141fab408dac6951d387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:05.645577 30437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a05f2fb56f0491cb87934d54fc3d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:05.645586 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.622318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41185 (local address 127.0.0.1:57306)
0812 06:52:05.637476 (+ 15158us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.637496 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.637600 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.638296 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.638300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.638323 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.638768 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.638784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.640236 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.640242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.643187 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.643203 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.644843 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.644849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.645003 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.645033 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.645113 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.645225 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15052,"mutex_wait_us":22,"thread_start_us":16053,"threads_started":1}
I20260812 06:52:05.646181 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.622463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.17.69.67:41185)
0812 06:52:05.637993 (+ 15530us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.637999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.638018 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:52:05.638084 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.638088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.638161 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.638261 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.638954 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.640071 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.643370 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.644501 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.644621 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.644716 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.645776 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.645913 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15411,"thread_start_us":15312,"threads_started":1}
I20260812 06:52:05.646996 30976 tablet_service.cc:1467] Processing CreateTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 (DEFAULT_TABLE table=test-table [id=1a4327d66cc141fab408dac6951d387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:05.647385 30976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a05f2fb56f0491cb87934d54fc3d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:05.650749 31270 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Bootstrap starting.
I20260812 06:52:05.652297 31270 tablet_bootstrap.cc:654] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:05.653707 31273 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Bootstrap starting.
I20260812 06:52:05.653801 31274 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Bootstrap starting.
I20260812 06:52:05.655243 31274 tablet_bootstrap.cc:654] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:05.655737 31273 tablet_bootstrap.cc:654] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:05.659834 31270 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: No bootstrap required, opened a new log
I20260812 06:52:05.659970 31270 ts_tablet_manager.cc:1397] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260812 06:52:05.660912 31273 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: No bootstrap required, opened a new log
I20260812 06:52:05.660723 31270 raft_consensus.cc:348] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.660989 31270 raft_consensus.cc:374] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:05.661036 31274 tablet_bootstrap.cc:492] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: No bootstrap required, opened a new log
I20260812 06:52:05.661096 31270 raft_consensus.cc:729] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef735495ae43443aa42cca77b43ea95a, State: Initialized, Role: FOLLOWER
I20260812 06:52:05.661181 31273 ts_tablet_manager.cc:1397] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260812 06:52:05.661186 31274 ts_tablet_manager.cc:1397] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260812 06:52:05.661387 31270 consensus_queue.cc:260] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.661885 31274 raft_consensus.cc:348] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.661950 31273 raft_consensus.cc:348] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.662117 31274 raft_consensus.cc:374] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:05.662191 31273 raft_consensus.cc:374] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:05.662209 31274 raft_consensus.cc:729] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc3faf0ee1c4e459e3a6b3e0cd03595, State: Initialized, Role: FOLLOWER
I20260812 06:52:05.662485 31274 consensus_queue.cc:260] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.662297 31273 raft_consensus.cc:729] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973e161db5644dd2b5c260e6129d7f2a, State: Initialized, Role: FOLLOWER
I20260812 06:52:05.662959 31273 consensus_queue.cc:260] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.663368 31274 ts_tablet_manager.cc:1428] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 06:52:05.664326 31273 ts_tablet_manager.cc:1428] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:52:05.670426 30559 heartbeater.cc:503] Master 127.17.69.125:36565 was elected leader, sending a full tablet report...
I20260812 06:52:05.672777 31100 heartbeater.cc:503] Master 127.17.69.125:36565 was elected leader, sending a full tablet report...
I20260812 06:52:05.673663 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.665004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:49319)
0812 06:52:05.673177 (+ 8173us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.673397 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":1520,"threads_started":1}
I20260812 06:52:05.674909 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.663733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:49847)
0812 06:52:05.674522 (+ 10789us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.674709 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10663,"thread_start_us":7257,"threads_started":1}
I20260812 06:52:05.676113 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.674249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:37663)
0812 06:52:05.675866 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.675951 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1527,"threads_started":1}
I20260812 06:52:05.676491 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.675605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:53373)
0812 06:52:05.676323 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.676387 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617}
I20260812 06:52:05.677203 31270 ts_tablet_manager.cc:1428] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Time spent starting tablet: real 0.017s user 0.007s sys 0.010s
I20260812 06:52:05.678488 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.677357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:57767)
0812 06:52:05.677900 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.678209 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":268,"threads_started":1}
I20260812 06:52:05.679654 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.679293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:48339)
0812 06:52:05.679423 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.679503 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 06:52:05.679752 30842 heartbeater.cc:503] Master 127.17.69.125:36565 was elected leader, sending a full tablet report...
I20260812 06:52:05.738304 31291 raft_consensus.cc:482] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:05.738574 31291 raft_consensus.cc:504] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.740068 31291 leader_election.cc:288] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363), 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
I20260812 06:52:05.747885 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.740833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:42507 (local address 127.17.69.67:41185)
0812 06:52:05.741006 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.741011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.741027 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:52:05.741104 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.741109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.741178 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.741262 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.741873 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.742979 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.745648 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.746884 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.747033 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.747105 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.747498 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.747626 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:52:05.747877 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.740395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:56703 (local address 127.17.69.66:36363)
0812 06:52:05.740576 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.740580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.740598 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:52:05.740982 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.740987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.741081 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.741200 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:05.741873 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.742979 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.745644 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.746871 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.747032 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.747105 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.747498 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.747626 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:52:05.747956 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.740717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41185 (local address 127.17.69.65:42507)
0812 06:52:05.740876 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.740889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.740980 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.741303 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.741307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.741329 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.741748 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.741762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.743101 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.743105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.745477 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.745493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.747107 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.747113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.747357 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.747383 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.747451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.747655 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":202}
I20260812 06:52:05.748328 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.740291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:36363 (local address 127.17.69.65:56703)
0812 06:52:05.740477 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.740496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:05.740645 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:05.741203 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:05.741206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:05.741233 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:05.741704 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.741722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.743098 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.743104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.745477 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:05.745493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.747064 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.747073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:05.747312 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.747362 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 06:52:05.747451 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:52:05.747655 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":147}
I20260812 06:52:05.748718 31009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a05f2fb56f0491cb87934d54fc3d922" candidate_uuid: "973e161db5644dd2b5c260e6129d7f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" is_pre_election: true
I20260812 06:52:05.748986 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a05f2fb56f0491cb87934d54fc3d922" candidate_uuid: "973e161db5644dd2b5c260e6129d7f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef735495ae43443aa42cca77b43ea95a" is_pre_election: true
I20260812 06:52:05.749143 31009 raft_consensus.cc:2454] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973e161db5644dd2b5c260e6129d7f2a in term 0.
I20260812 06:52:05.749225 30754 raft_consensus.cc:2454] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973e161db5644dd2b5c260e6129d7f2a in term 0.
I20260812 06:52:05.749667 30349 leader_election.cc:302] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973e161db5644dd2b5c260e6129d7f2a, ef735495ae43443aa42cca77b43ea95a; no voters:
I20260812 06:52:05.749985 31291 raft_consensus.cc:2781] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:05.750102 31291 raft_consensus.cc:482] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:05.750193 31291 raft_consensus.cc:3037] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:05.752377 31291 raft_consensus.cc:504] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.752887 31291 leader_election.cc:288] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [CANDIDATE]: Term 1 election: Requested vote from peers ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363), 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185)
I20260812 06:52:05.753407 30754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a05f2fb56f0491cb87934d54fc3d922" candidate_uuid: "973e161db5644dd2b5c260e6129d7f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef735495ae43443aa42cca77b43ea95a"
I20260812 06:52:05.753450 31009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a05f2fb56f0491cb87934d54fc3d922" candidate_uuid: "973e161db5644dd2b5c260e6129d7f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595"
I20260812 06:52:05.753628 30754 raft_consensus.cc:3037] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:05.753628 31009 raft_consensus.cc:3037] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:05.756517 30754 raft_consensus.cc:2454] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973e161db5644dd2b5c260e6129d7f2a in term 1.
I20260812 06:52:05.756526 31009 raft_consensus.cc:2454] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973e161db5644dd2b5c260e6129d7f2a in term 1.
I20260812 06:52:05.756898 30349 leader_election.cc:302] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973e161db5644dd2b5c260e6129d7f2a, ef735495ae43443aa42cca77b43ea95a; no voters:
I20260812 06:52:05.757171 31291 raft_consensus.cc:2781] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:05.757601 31291 raft_consensus.cc:686] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 1 LEADER]: Becoming Leader. State: Replica: 973e161db5644dd2b5c260e6129d7f2a, State: Running, Role: LEADER
I20260812 06:52:05.757977 31291 consensus_queue.cc:237] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } }
I20260812 06:52:05.758613 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.758132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:43697)
0812 06:52:05.758313 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.758447 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:52:05.761313 29846 catalog_manager.cc:5495] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65). New cstate: current_term: 1 leader_uuid: "973e161db5644dd2b5c260e6129d7f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "973e161db5644dd2b5c260e6129d7f2a" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260812 06:52:05.784298 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.779634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.17.69.65:46729)
0812 06:52:05.779802 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.779806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.779824 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:52:05.780029 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.780032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.780122 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.780240 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:05.780528 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.781230 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.782433 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.782839 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.782927 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.783346 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:05.783849 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:05.783958 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.784016 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.784078 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:52:05.787967 30754 raft_consensus.cc:1261] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Refusing update from remote peer 973e161db5644dd2b5c260e6129d7f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:05.788154 31009 raft_consensus.cc:1261] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Refusing update from remote peer 973e161db5644dd2b5c260e6129d7f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:05.788630 31298 consensus_queue.cc:1035] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef735495ae43443aa42cca77b43ea95a" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 36363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:05.789062 31291 consensus_queue.cc:1035] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc3faf0ee1c4e459e3a6b3e0cd03595" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:05.791930 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.791451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:40651)
0812 06:52:05.791613 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.791736 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:52:05.792835 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.791038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:51101)
0812 06:52:05.792524 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.792649 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1356}
I20260812 06:52:05.796260 31306 mvcc.cc:205] Tried to move back new op lower bound from 7317575785621381120 to 7317575785508687872. Current Snapshot: MvccSnapshot[applied={T|T < 7317575785621381120}]
I20260812 06:52:05.796530 31305 mvcc.cc:205] Tried to move back new op lower bound from 7317575785621381120 to 7317575785508687872. Current Snapshot: MvccSnapshot[applied={T|T < 7317575785621381120}]
I20260812 06:52:05.797628 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.796987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.66:52705)
0812 06:52:05.797378 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.797467 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267}
I20260812 06:52:05.798540 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.797959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:56181)
0812 06:52:05.798257 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.798365 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260812 06:52:05.803375 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.802100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.65:53949)
0812 06:52:05.803132 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.803213 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":932}
I20260812 06:52:05.803397 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.802912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.17.69.67:40547)
0812 06:52:05.803135 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:05.803231 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 06:52:05.811085 29841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:05.811367 29841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:05.817279 29841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 283f83cd6ce041b49d02336f4773bf4a: Sending DeleteTablet for 3 replicas of tablet 8a05f2fb56f0491cb87934d54fc3d922
I20260812 06:52:05.818385 30437 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:05 UTC) from {username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:32782
I20260812 06:52:05.818480 30719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:05 UTC) from {username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:33510
I20260812 06:52:05.818817 30976 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a05f2fb56f0491cb87934d54fc3d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:05 UTC) from {username='oryx', principal='oryx/127.17.69.125@KRBTEST.COM'} at 127.0.0.1:57306
I20260812 06:52:05.819149 31326 tablet_replica.cc:330] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: stopping tablet replica
I20260812 06:52:05.819360 31320 tablet_replica.cc:330] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: stopping tablet replica
I20260812 06:52:05.819350 31326 raft_consensus.cc:2229] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:05.819664 31326 raft_consensus.cc:2258] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:05.819646 31320 raft_consensus.cc:2229] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:05.820358 31320 raft_consensus.cc:2258] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:05.820734 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 30178
I20260812 06:52:05.821095 31326 ts_tablet_manager.cc:1905] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:05.822113 31320 ts_tablet_manager.cc:1905] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:05.827517 31326 ts_tablet_manager.cc:1918] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:05.827646 31326 log.cc:1196] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/8a05f2fb56f0491cb87934d54fc3d922
I20260812 06:52:05.828111 31326 ts_tablet_manager.cc:1939] T 8a05f2fb56f0491cb87934d54fc3d922 P ef735495ae43443aa42cca77b43ea95a: Deleting consensus metadata
I20260812 06:52:05.829448 31320 ts_tablet_manager.cc:1918] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:05.829650 31320 log.cc:1196] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/8a05f2fb56f0491cb87934d54fc3d922
I20260812 06:52:05.829638 29819 catalog_manager.cc:4841] TS ef735495ae43443aa42cca77b43ea95a (127.17.69.66:36363): tablet 8a05f2fb56f0491cb87934d54fc3d922 (table test-table [id=1a4327d66cc141fab408dac6951d387a]) successfully deleted
I20260812 06:52:05.830217 31320 ts_tablet_manager.cc:1939] T 8a05f2fb56f0491cb87934d54fc3d922 P 973e161db5644dd2b5c260e6129d7f2a: Deleting consensus metadata
I20260812 06:52:05.831350 29819 catalog_manager.cc:4841] TS 973e161db5644dd2b5c260e6129d7f2a (127.17.69.65:46729): tablet 8a05f2fb56f0491cb87934d54fc3d922 (table test-table [id=1a4327d66cc141fab408dac6951d387a]) successfully deleted
I20260812 06:52:05.834842 31328 tablet_replica.cc:330] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: stopping tablet replica
I20260812 06:52:05.835067 31328 raft_consensus.cc:2229] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:05.835384 31328 raft_consensus.cc:2258] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:05.836237 31328 ts_tablet_manager.cc:1905] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:05.841449 31328 ts_tablet_manager.cc:1918] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:05.841575 31328 log.cc:1196] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/8a05f2fb56f0491cb87934d54fc3d922
I20260812 06:52:05.841933 31328 ts_tablet_manager.cc:1939] T 8a05f2fb56f0491cb87934d54fc3d922 P 7bc3faf0ee1c4e459e3a6b3e0cd03595: Deleting consensus metadata
I20260812 06:52:05.842902 29819 catalog_manager.cc:4841] TS 7bc3faf0ee1c4e459e3a6b3e0cd03595 (127.17.69.67:41185): tablet 8a05f2fb56f0491cb87934d54fc3d922 (table test-table [id=1a4327d66cc141fab408dac6951d387a]) successfully deleted
I20260812 06:52:05.854370 31266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.822622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.17.69.65:46729)
0812 06:52:05.822793 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.822799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.834917 (+ 12118us) server_negotiation.cc:407] Connection header received
0812 06:52:05.835098 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.835103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.835181 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.835292 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:05.837430 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.838653 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.840574 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.841277 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.841412 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.848554 (+ 7142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:05.848566 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:05.848581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:05.848611 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:05.851684 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:05.852480 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:05.852487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:05.852489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:05.852642 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:05.853027 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:05.853030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:05.853032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:05.853360 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:05.853587 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.853906 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.854055 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:52:05.855194 30547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:05.956664 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 30178
I20260812 06:52:05.979562 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 30578
I20260812 06:52:05.992643 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:05.980314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.17.69.66:36363)
0812 06:52:05.980495 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:05.980500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:05.980583 (+ 83us) server_negotiation.cc:407] Connection header received
0812 06:52:05.980769 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:05.980774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:05.980843 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:05.980924 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:05.982221 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.983046 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.984253 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:05.984670 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:05.984777 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:05.987282 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:05.987298 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:05.987314 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:05.987360 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:05.989975 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:05.990669 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:05.990677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:05.990680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:05.990806 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:05.991164 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:05.991170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:05.991174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:05.991540 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:05.991867 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:05.992116 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 06:52:05.992338 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:52:05.993276 30828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:06.085755 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 30578
I20260812 06:52:06.108893 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 30862
I20260812 06:52:06.121461 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:06.109670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.17.69.67:41185)
0812 06:52:06.109855 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:06.109860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:06.109879 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:52:06.109977 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:06.109980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:06.110052 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:06.110131 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:06.111500 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:06.112285 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:06.113537 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:06.113946 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:06.114038 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:06.116548 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:06.116563 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:06.116580 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:06.116611 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:06.119183 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:06.119774 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:06.119779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:06.119781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:06.119892 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:06.120181 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:06.120184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:06.120186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:06.120505 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:06.120719 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:06.120941 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 06:52:06.121114 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:52:06.122123 31085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:06.205431 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:06.204740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35176)
0812 06:52:06.205177 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:06.205278 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":227,"threads_started":1}
I20260812 06:52:06.206717 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:06.206390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45281 (local address 127.0.0.1:35190)
0812 06:52:06.206524 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:06.206610 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:45281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 06:52:06.244915 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 30862
I20260812 06:52:06.273300 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 29600
I20260812 06:52:06.274365 29904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:06.392993 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 29600
W20260812 06:52:06.419063 30195 connection.cc:535] server connection from 127.0.0.1:52530 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:52:06.419428 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 29785
I20260812 06:52:06.420439 30271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:06.532986 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 29785
2026-08-12T06:52:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9526 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:52:06 dist-test-slave-6zbq krb5kdc[31361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:52:06 dist-test-slave-6zbq krb5kdc[31361](info): set up 2 sockets
Aug 12 06:52:06 dist-test-slave-6zbq krb5kdc[31361](info): commencing operation
krb5kdc: starting...
W20260812 06:52:07.764058 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:52:07 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab.
Entry for principal oryx/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab.
I20260812 06:52:07.973605 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36505
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab
--principal=oryx/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:08.278496 31384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:08.278729 31384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:08.278849 31384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:08.285746 31384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:08.285833 31384 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 06:52:08.285897 31384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:08.285974 31384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:08.286031 31384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:08.286118 31384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:08.295217 31384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:36505
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.126.keytab
--principal=oryx/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:08.296239 31384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:08.297398 31384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:08.307984 31394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:08.308027 31393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:08.308065 31396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:08.308840 31384 server_base.cc:1034] running on GCE node
I20260812 06:52:08.309514 31384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:08.310773 31384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:08.311978 31384 hybrid_clock.cc:648] HybridClock initialized: now 1786517528311918 us; error 30 us; skew 500 ppm
I20260812 06:52:08.312274 31384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:08 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517528, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:08.315083 31384 init.cc:377] Logged in from keytab as oryx/127.17.69.126@KRBTEST.COM (short username oryx)
I20260812 06:52:08.316442 31384 webserver.cc:460] Webserver started at http://127.17.69.126:36291/ using document root <none> and password file <none>
I20260812 06:52:08.317047 31384 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:08.317143 31384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:08.317445 31384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:08.320205 31384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "36a64d23dae44bb0af089be27ed7adce"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "49615c9dd0d025ba07959844d2a30761"
server_key_iv: "0c511156b398b12b6f5a127c3477cf8f"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.320950 31384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "36a64d23dae44bb0af089be27ed7adce"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "49615c9dd0d025ba07959844d2a30761"
server_key_iv: "0c511156b398b12b6f5a127c3477cf8f"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.325528 31384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 06:52:08.328646 31407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:08.329777 31384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:08.329919 31384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "36a64d23dae44bb0af089be27ed7adce"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "49615c9dd0d025ba07959844d2a30761"
server_key_iv: "0c511156b398b12b6f5a127c3477cf8f"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.330098 31384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:08.368958 31384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:08.372612 31384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:08.372844 31384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:08.392269 31384 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:36505
I20260812 06:52:08.392304 31506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:36505 every 8 connection(s)
I20260812 06:52:08.393443 31384 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:08.396821 31507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:08.400954 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 31384
I20260812 06:52:08.401191 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:52:08.401988 17685 external_mini_cluster.cc:1428] Setting key 634b76b7fafa0f902dbfb26ef8892d4b
I20260812 06:52:08.407924 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Bootstrap starting.
I20260812 06:52:08.411302 31507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:08.412402 31507 log.cc:824] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:08.414832 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: No bootstrap required, opened a new log
Aug 12 06:52:08 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:08.423319 31507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } }
I20260812 06:52:08.423642 31507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:08.423727 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:08.403900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.17.69.126:36505)
0812 06:52:08.404591 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:08.404601 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:08.404644 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:52:08.406606 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:08.406626 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:08.406950 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:08.407234 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:08.409022 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.410272 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.411924 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.412594 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.412724 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:08.416882 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:08.416890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:08.416902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:08.416935 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:08.419861 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.420628 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.420646 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.420651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.420816 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.421215 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.421218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.421220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.421734 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:08.421967 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:08.422423 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 06:52:08.422785 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":328,"threads_started":1}
I20260812 06:52:08.423751 31507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a64d23dae44bb0af089be27ed7adce, State: Initialized, Role: FOLLOWER
I20260812 06:52:08.424327 31507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } }
I20260812 06:52:08.424553 31507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:08.424674 31507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:08.424826 31507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:08.427512 31507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } }
I20260812 06:52:08.427947 31507 leader_election.cc:302] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a64d23dae44bb0af089be27ed7adce; no voters:
I20260812 06:52:08.428416 31507 leader_election.cc:288] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:52:08.428525 31514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:08.429653 31514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [term 1 LEADER]: Becoming Leader. State: Replica: 36a64d23dae44bb0af089be27ed7adce, State: Running, Role: LEADER
I20260812 06:52:08.430164 31514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } }
I20260812 06:52:08.430370 31507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:08.432785 31515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a64d23dae44bb0af089be27ed7adce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } } }
I20260812 06:52:08.433195 31516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a64d23dae44bb0af089be27ed7adce. Latest consensus state: current_term: 1 leader_uuid: "36a64d23dae44bb0af089be27ed7adce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a64d23dae44bb0af089be27ed7adce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36505 } } }
I20260812 06:52:08.433629 31516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:08.433946 31515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:08.435410 31529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:08.440600 31529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:08.448697 31529 catalog_manager.cc:1347] Generated new cluster ID: e5f0966471cd4f0c8f5cfad5236417db
I20260812 06:52:08.448794 31529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:08.476858 31529 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:52:08.478010 31529 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:08.483129 31529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Generated new TSK 0
I20260812 06:52:08.483821 31529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
I20260812 06:52:08.539327 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36505
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:08.859081 31553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:08.859328 31553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:08.859450 31553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:08.866742 31553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:08.866832 31553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:08.866947 31553 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:52:08.867048 31553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:08.876410 31553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:08.877461 31553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:08.878710 31553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:08.890748 31566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:08.890791 31568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:08.890712 31571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:08.891494 31553 server_base.cc:1034] running on GCE node
I20260812 06:52:08.892055 31553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:08.892864 31553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:08.894073 31553 hybrid_clock.cc:648] HybridClock initialized: now 1786517528894008 us; error 41 us; skew 500 ppm
I20260812 06:52:08.894330 31553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:08 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517528, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:08.897275 31553 init.cc:377] Logged in from keytab as oryx/127.17.69.65@KRBTEST.COM (short username oryx)
I20260812 06:52:08.898577 31553 webserver.cc:460] Webserver started at http://127.17.69.65:34973/ using document root <none> and password file <none>
I20260812 06:52:08.899175 31553 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:08.899272 31553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:08.899575 31553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:08.902242 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "27c3d4d137c0e71c460bc81fbfde8ec5"
server_key_iv: "d23afa425228aa39556df92fec544891"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.903026 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "27c3d4d137c0e71c460bc81fbfde8ec5"
server_key_iv: "d23afa425228aa39556df92fec544891"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.907740 31553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:52:08.910933 31582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:08.912103 31553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:52:08.912243 31553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
format_stamp: "Formatted at 2026-08-12 06:52:08 on dist-test-slave-6zbq"
server_key: "27c3d4d137c0e71c460bc81fbfde8ec5"
server_key_iv: "d23afa425228aa39556df92fec544891"
server_key_version: "encryptionkey@0"
I20260812 06:52:08.912433 31553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:08.926762 31553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:08.930650 31553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:08.930869 31553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:08.933367 31553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:08.933465 31553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:08.933643 31553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:08.933760 31553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 06:52:08 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517528, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:08.954164 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:08.933772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:46031 (local address 127.17.69.126:36505)
0812 06:52:08.934251 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:08.934258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:08.935326 (+ 1068us) server_negotiation.cc:407] Connection header received
0812 06:52:08.937219 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:08.937226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:08.937309 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:08.937422 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:08.939667 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.940560 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.942918 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.943594 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.943729 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:08.948163 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:08.948170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:08.948173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:08.948207 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:08.951245 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.951920 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.951926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.951930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.952083 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.952478 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.952482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.952486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.952839 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:08.953003 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:08.953534 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 06:52:08.953769 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":227,"threads_started":1}
I20260812 06:52:08.954734 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:08.934461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.65:46031)
0812 06:52:08.935169 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:08.935212 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:08.936951 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:08.937694 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:08.937706 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:08.938480 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:08.939457 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:08.939491 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.940771 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.940776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:08.942760 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:08.942777 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.943814 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.943819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:08.943993 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:08.944607 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:08.944650 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:08.947908 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:08.951400 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:08.951409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:08.951420 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:08.951804 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:08.952187 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:08.952190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:08.952193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:08.952378 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:08.952949 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:08.952953 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:08.953217 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:52:08.953416 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 06:52:08.953738 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":291,"threads_started":1}
I20260812 06:52:08.973318 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:08.960783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:55037 (local address 127.17.69.126:36505)
0812 06:52:08.960959 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:08.960965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:08.961734 (+ 769us) server_negotiation.cc:407] Connection header received
0812 06:52:08.961873 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:08.961876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:08.961929 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:08.961998 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:08.963328 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.964144 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.965407 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.965830 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.965933 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:08.967736 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:08.967744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:08.967748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:08.967785 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:08.970429 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.971292 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.971298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.971301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.971452 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:08.971849 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:08.971855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:08.971858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:08.972220 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:08.972365 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:08.972795 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 06:52:08.972984 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:52:08.973423 31703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:08.960982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.65:55037)
0812 06:52:08.961629 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:08.961648 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:08.961781 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:08.962038 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:08.962043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:08.962653 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:08.963172 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:08.963190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.964285 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.964293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:08.965280 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:08.965292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:08.965999 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:08.966006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:08.966148 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:08.966671 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:08.966694 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:08.967518 (+ 824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:08.970574 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:08.970577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:08.970580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:08.971116 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:08.971546 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:08.971555 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:08.971559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:08.971731 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:08.972341 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:08.972346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:08.972551 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 06:52:08.972689 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 06:52:08.973004 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":332,"threads_started":1}
I20260812 06:52:08.984095 31553 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:40477
I20260812 06:52:08.984125 31814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:40477 every 8 connection(s)
I20260812 06:52:08.985131 31553 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:08.986851 31815 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36505
I20260812 06:52:08.987049 31815 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:08.987612 31815 heartbeater.cc:511] Master 127.17.69.126:36505 requested a full tablet report, sending...
I20260812 06:52:08.989356 31431 ts_manager.cc:194] Registered new tserver with Master: bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477)
I20260812 06:52:08.990793 31431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:46031
I20260812 06:52:08.991195 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 31553
I20260812 06:52:08.991405 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:52:08.992174 17685 external_mini_cluster.cc:1428] Setting key 0de9fefb1deacd366c21e23595f4a4ef
WARNING: no policy specified for oryx/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
I20260812 06:52:09.041271 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36505
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:09.349864 31822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:09.350091 31822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:09.350193 31822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:09.357220 31822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:09.357316 31822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:09.357419 31822 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:52:09.357502 31822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:09.366514 31822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:09.367516 31822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:09.368665 31822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:09.380142 31834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:09.380240 31830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:09.380264 31839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:09.381189 31822 server_base.cc:1034] running on GCE node
I20260812 06:52:09.381703 31822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:09.382529 31822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:09.383738 31822 hybrid_clock.cc:648] HybridClock initialized: now 1786517529383676 us; error 39 us; skew 500 ppm
I20260812 06:52:09.383986 31822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:09 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517529, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:09.386905 31822 init.cc:377] Logged in from keytab as oryx/127.17.69.66@KRBTEST.COM (short username oryx)
I20260812 06:52:09.388113 31822 webserver.cc:460] Webserver started at http://127.17.69.66:42967/ using document root <none> and password file <none>
I20260812 06:52:09.388710 31822 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:09.388806 31822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:09.389096 31822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:09.391974 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "827a2985832f2c32072ccce38e432752"
server_key_iv: "748bf4b7fa45fa39876546d455cfbabf"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.392933 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "827a2985832f2c32072ccce38e432752"
server_key_iv: "748bf4b7fa45fa39876546d455cfbabf"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.398765 31822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260812 06:52:09.402633 31853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:09.404035 31822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:52:09.404189 31822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "827a2985832f2c32072ccce38e432752"
server_key_iv: "748bf4b7fa45fa39876546d455cfbabf"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.404419 31822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:09.425184 31822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:09.428934 31822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:09.429138 31822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:09.431687 31822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:09.431792 31822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:09.431943 31822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:09.432013 31822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:52:09 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517529, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:09.452688 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.431979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:49675 (local address 127.17.69.126:36505)
0812 06:52:09.432175 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:09.432181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:09.433505 (+ 1324us) server_negotiation.cc:407] Connection header received
0812 06:52:09.435599 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:09.435607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:09.435696 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:09.435815 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:09.437705 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.438869 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.441138 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.441714 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.441826 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.445953 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:09.445961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:09.445966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:09.446009 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:09.449367 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.450237 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.450242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.450244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.450412 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.450976 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.450982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.450985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.451318 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:09.451469 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:09.452285 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 06:52:09.452453 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:52:09.454114 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.432486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.66:49675)
0812 06:52:09.433336 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:09.433378 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:09.435259 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:09.436074 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:09.436085 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:09.436774 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:09.437511 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.437536 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.439026 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.439031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.440987 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.441000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.441960 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.441968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.442242 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.442872 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:09.442905 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:09.445714 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:09.449511 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.449519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.449534 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.450077 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.450520 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.450525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.450530 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.450789 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.451493 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:09.451502 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:09.451925 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 06:52:09.452231 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 06:52:09.452716 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":359,"threads_started":1}
I20260812 06:52:09.473008 31822 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:40187
I20260812 06:52:09.473031 32062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:40187 every 8 connection(s)
I20260812 06:52:09.474269 31822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:09.476693 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.460850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37429 (local address 127.17.69.126:36505)
0812 06:52:09.461040 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:09.461046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:09.463269 (+ 2223us) server_negotiation.cc:407] Connection header received
0812 06:52:09.463508 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:09.463514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:09.463590 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:09.463698 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:09.464794 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.465594 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.467399 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.467931 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.468058 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.470284 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:09.470290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:09.470294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:09.470329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:09.473131 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.473786 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.473792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.473795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.473949 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.474284 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.474289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.474293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.474722 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:09.474872 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:09.476254 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0812 06:52:09.476442 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:52:09.475921 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.460808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.66:37429)
0812 06:52:09.463155 (+ 2347us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:09.463176 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:09.463326 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:09.463730 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:09.463735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:09.464359 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:09.464658 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.464668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.465723 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.465732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.467270 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.467285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.468163 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.468175 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.468375 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.469032 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:09.469060 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:09.470088 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:09.473283 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.473286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.473289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.473663 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.474057 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.474060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.474062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.474180 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.474866 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:09.474872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:09.475082 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 06:52:09.475253 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 06:52:09.475581 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2166,"thread_start_us":2007,"threads_started":1}
I20260812 06:52:09.476390 32071 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36505
I20260812 06:52:09.477020 32071 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:09.477684 32071 heartbeater.cc:511] Master 127.17.69.126:36505 requested a full tablet report, sending...
I20260812 06:52:09.478987 31431 ts_manager.cc:194] Registered new tserver with Master: dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:09.479758 31431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:49675
I20260812 06:52:09.481467 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 31822
I20260812 06:52:09.481613 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:52:09.482002 17685 external_mini_cluster.cc:1428] Setting key a85003afa90506182d06e6c9a4690d78
WARNING: no policy specified for oryx/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
I20260812 06:52:09.531383 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36505
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:09.843695 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:09.843953 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:09.844087 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:09.850980 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:09.851085 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:09.851210 32077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 06:52:09.851317 32077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:09.860590 32077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.69.126:36505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:09.861725 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:09.863109 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:09.874889 32083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:09.875030 32089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:09.875054 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:09.876088 32077 server_base.cc:1034] running on GCE node
I20260812 06:52:09.876638 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:09.877435 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:09.878634 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786517529878570 us; error 35 us; skew 500 ppm
I20260812 06:52:09.878885 32077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:09 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517529, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:09.881703 32077 init.cc:377] Logged in from keytab as oryx/127.17.69.67@KRBTEST.COM (short username oryx)
I20260812 06:52:09.883008 32077 webserver.cc:460] Webserver started at http://127.17.69.67:42103/ using document root <none> and password file <none>
I20260812 06:52:09.883612 32077 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:09.883721 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:09.884011 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:09.886816 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "42658cf367ee485da2770938e977934c"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "d7e02a8b18148500c5186ed405087d47"
server_key_iv: "fb4e1a0f6778412c746768c8347386d5"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.887535 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "42658cf367ee485da2770938e977934c"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "d7e02a8b18148500c5186ed405087d47"
server_key_iv: "fb4e1a0f6778412c746768c8347386d5"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.892019 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:52:09.895134 32106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:09.896446 32077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:52:09.896607 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "42658cf367ee485da2770938e977934c"
format_stamp: "Formatted at 2026-08-12 06:52:09 on dist-test-slave-6zbq"
server_key: "d7e02a8b18148500c5186ed405087d47"
server_key_iv: "fb4e1a0f6778412c746768c8347386d5"
server_key_version: "encryptionkey@0"
I20260812 06:52:09.896806 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:09.910092 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:09.913496 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:09.913719 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:09.916203 32077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:09.916299 32077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:09.916462 32077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:09.916529 32077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:52:09 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517529, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:09.938582 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.916518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:53133 (local address 127.17.69.126:36505)
0812 06:52:09.916731 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:09.916737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:09.918095 (+ 1358us) server_negotiation.cc:407] Connection header received
0812 06:52:09.920764 (+ 2669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:09.920771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:09.920859 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:09.920968 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:09.923271 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.924263 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.926527 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.927113 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.927229 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.931822 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:09.931830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:09.931834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:09.931879 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:09.934948 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.935836 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.935843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.935847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.936035 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.936567 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.936573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.936577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.936845 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:09.936981 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:09.937911 (+ 930us) server_negotiation.cc:299] Negotiation successful
0812 06:52:09.938161 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:52:09.939386 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.917058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.67:53133)
0812 06:52:09.917917 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:09.917965 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:09.920457 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:09.921259 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:09.921270 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:09.922148 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:09.923066 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.923093 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.924397 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.924403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.926303 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.926316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.927533 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.927541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.927749 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.928479 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:09.928525 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:09.931530 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:09.935071 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.935079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.935097 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.935697 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.936169 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.936173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.936177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.936442 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.936978 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:09.936985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:09.937426 (+ 441us) client_negotiation.cc:769] Sending connection context
0812 06:52:09.937726 (+ 300us) client_negotiation.cc:240] Negotiation successful
0812 06:52:09.938198 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":345,"threads_started":1}
I20260812 06:52:09.961535 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.948094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:49095 (local address 127.17.69.126:36505)
0812 06:52:09.948229 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:09.948233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:09.948854 (+ 621us) server_negotiation.cc:407] Connection header received
0812 06:52:09.948985 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:09.948991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:09.949045 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:09.949121 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:09.950035 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.950909 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.952855 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.953261 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.953353 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.955556 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:09.955561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:09.955565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:09.955594 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:09.958830 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.959659 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.959665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.959670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.959806 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:09.960138 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:09.960144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:09.960148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:09.960431 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:09.960559 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:09.960954 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 06:52:09.961203 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 06:52:09.961612 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:09.948192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.17.69.67:49095)
0812 06:52:09.948759 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:09.948777 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:09.948891 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:09.949155 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:09.949159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:09.949579 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:09.949898 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.949908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.951036 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.951046 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.952707 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:09.952730 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:09.953431 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:09.953437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:09.953626 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:09.954188 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:09.954209 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:09.955371 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:09.958937 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.958943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.958947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.959547 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.959907 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:09.959912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:09.959916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:09.960039 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:09.960546 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:09.960550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:09.960720 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 06:52:09.960836 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 06:52:09.961153 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":329,"threads_started":1}
I20260812 06:52:09.965878 32077 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41203
I20260812 06:52:09.965900 32303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41203 every 8 connection(s)
I20260812 06:52:09.967088 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:09.969031 32305 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36505
I20260812 06:52:09.969242 32305 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:09.969848 32305 heartbeater.cc:511] Master 127.17.69.126:36505 requested a full tablet report, sending...
I20260812 06:52:09.971128 31431 ts_manager.cc:194] Registered new tserver with Master: 42658cf367ee485da2770938e977934c (127.17.69.67:41203)
I20260812 06:52:09.971704 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 32077
I20260812 06:52:09.971899 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:09.972213 31431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:53133
I20260812 06:52:09.972509 17685 external_mini_cluster.cc:1428] Setting key fdca00a1323eaf2aef3244fe2f22576d
I20260812 06:52:09.975383 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:52:09.993695 31815 heartbeater.cc:503] Master 127.17.69.126:36505 was elected leader, sending a full tablet report...
Aug 12 06:52:10 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 06:52:10 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:10 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:10.481813 32071 heartbeater.cc:503] Master 127.17.69.126:36505 was elected leader, sending a full tablet report...
I20260812 06:52:10.974766 32305 heartbeater.cc:503] Master 127.17.69.126:36505 was elected leader, sending a full tablet report...
W20260812 06:52:11.015515 32317 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.987s user 0.004s sys 0.015s
W20260812 06:52:11.016650 31603 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:10.022900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.17.69.126:36505)
0812 06:52:10.023144 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:10.023151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:10.023389 (+ 238us) server_negotiation.cc:407] Connection header received
0812 06:52:10.023572 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:10.023578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:10.023646 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:10.023733 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:10.025097 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:10.025848 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:10.027096 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:10.027525 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:10.027639 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.016319 (+988680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42908: BlockingRecv error: recv got EOF from 127.0.0.1:42908 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 06:52:11 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517530, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:11.038614 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.020330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.17.69.126:36505)
0812 06:52:11.020489 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.020493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.020909 (+ 416us) server_negotiation.cc:407] Connection header received
0812 06:52:11.021037 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.021041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.021141 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.021223 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:11.022447 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.023169 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.024354 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.024762 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.024847 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.033985 (+ 9138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:11.033991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:11.033994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:11.034027 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:11.036466 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:11.037042 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:11.037046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:11.037048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:11.037165 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:11.037447 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:11.037450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:11.037452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:11.037717 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:11.037927 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.038084 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.038268 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:52:11.042305 31431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:11.048872 31431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:11.080317 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.059709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.0.0.1:50826)
0812 06:52:11.071731 (+ 12022us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.071771 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.072029 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.073001 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.073009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.073043 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.073522 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.073536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.075448 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.075453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.077738 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.077751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.079353 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.079359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.079532 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.079572 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.079692 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.079874 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11733,"mutex_wait_us":1,"thread_start_us":14750,"threads_started":1}
I20260812 06:52:11.081153 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.059931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.17.69.67:41203)
0812 06:52:11.071115 (+ 11184us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.071124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.071872 (+ 748us) server_negotiation.cc:407] Connection header received
0812 06:52:11.072269 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.072278 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.072527 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.072737 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.073748 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.075258 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.077926 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.079153 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.079273 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.079475 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.080694 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.080857 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11021,"thread_start_us":10885,"threads_started":1}
I20260812 06:52:11.082505 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.060361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.17.69.65:40477)
0812 06:52:11.070517 (+ 10156us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.070527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.074436 (+ 3909us) server_negotiation.cc:407] Connection header received
0812 06:52:11.075011 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.075019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.075231 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.075380 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.076349 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.077722 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.080090 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.081151 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.081282 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.081485 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.082023 (+ 538us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.082204 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9956,"thread_start_us":10153,"threads_started":1}
I20260812 06:52:11.083122 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.060119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40477 (local address 127.0.0.1:33584)
0812 06:52:11.074305 (+ 14186us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.074322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.074829 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.075635 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.075639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.075667 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.076075 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.076087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.077882 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.077887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.079943 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.079955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.081648 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.081656 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.081817 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.081849 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.082693 (+ 844us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.082827 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14056,"thread_start_us":11134,"threads_started":1}
I20260812 06:52:11.084147 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.071990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40187 (local address 127.0.0.1:47028)
0812 06:52:11.076626 (+ 4636us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.076641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.076764 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.077300 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.077305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.077329 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.077780 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.077797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.079690 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.079695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.081739 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.081753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.083387 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.083394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.083549 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.083581 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.083680 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.083796 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4522,"mutex_wait_us":58,"thread_start_us":4388,"threads_started":1}
I20260812 06:52:11.084178 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.072295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.17.69.66:40187)
0812 06:52:11.072890 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.072899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.076742 (+ 3843us) server_negotiation.cc:407] Connection header received
0812 06:52:11.076901 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.076909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.077095 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.077293 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.077997 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.079554 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.081912 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.083089 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.083213 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.083396 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.083750 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.083904 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":292,"threads_started":1}
I20260812 06:52:11.085577 32186 tablet_service.cc:1467] Processing CreateTablet for tablet 758d645562cc401ebcc1dfc4969848f8 (DEFAULT_TABLE table=test-table [id=077d3c19e49a4510b9191f9c45a04c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.086803 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758d645562cc401ebcc1dfc4969848f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.086464 31682 tablet_service.cc:1467] Processing CreateTablet for tablet 758d645562cc401ebcc1dfc4969848f8 (DEFAULT_TABLE table=test-table [id=077d3c19e49a4510b9191f9c45a04c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.087873 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758d645562cc401ebcc1dfc4969848f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.087728 31937 tablet_service.cc:1467] Processing CreateTablet for tablet 758d645562cc401ebcc1dfc4969848f8 (DEFAULT_TABLE table=test-table [id=077d3c19e49a4510b9191f9c45a04c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.089013 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758d645562cc401ebcc1dfc4969848f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.098239 32350 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Bootstrap starting.
I20260812 06:52:11.101503 32350 tablet_bootstrap.cc:654] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.102779 32350 log.cc:824] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:11.107074 32351 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Bootstrap starting.
I20260812 06:52:11.110687 32351 tablet_bootstrap.cc:654] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.110960 32352 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Bootstrap starting.
I20260812 06:52:11.111846 32351 log.cc:824] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:11.113420 32350 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: No bootstrap required, opened a new log
I20260812 06:52:11.113684 32350 ts_tablet_manager.cc:1397] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260812 06:52:11.114321 32352 tablet_bootstrap.cc:654] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.114754 32351 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: No bootstrap required, opened a new log
I20260812 06:52:11.115005 32351 ts_tablet_manager.cc:1397] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260812 06:52:11.115331 32352 log.cc:824] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:11.118299 32352 tablet_bootstrap.cc:492] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: No bootstrap required, opened a new log
I20260812 06:52:11.118541 32352 ts_tablet_manager.cc:1397] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260812 06:52:11.123647 32350 raft_consensus.cc:348] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.124047 32350 raft_consensus.cc:374] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.124188 32350 raft_consensus.cc:729] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42658cf367ee485da2770938e977934c, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.124775 32350 consensus_queue.cc:260] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.124897 32351 raft_consensus.cc:348] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.125275 32351 raft_consensus.cc:374] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.125105 32352 raft_consensus.cc:348] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.125468 32351 raft_consensus.cc:729] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.125517 32352 raft_consensus.cc:374] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.125659 32352 raft_consensus.cc:729] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.126181 32351 consensus_queue.cc:260] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.126277 32352 consensus_queue.cc:260] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.126566 32350 ts_tablet_manager.cc:1428] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260812 06:52:11.128046 32351 ts_tablet_manager.cc:1428] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260812 06:52:11.128357 32352 ts_tablet_manager.cc:1428] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260812 06:52:11.157722 32361 raft_consensus.cc:482] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:11.157927 32361 raft_consensus.cc:504] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.159480 32361 leader_election.cc:288] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42658cf367ee485da2770938e977934c (127.17.69.67:41203), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:11.165700 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.159857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:52571 (local address 127.17.69.66:40187)
0812 06:52:11.160073 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.160080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.160790 (+ 710us) server_negotiation.cc:407] Connection header received
0812 06:52:11.160952 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.160958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.161025 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.161109 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.161693 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.162402 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.164121 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.164851 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.164988 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.165056 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.165350 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.165470 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:52:11.165733 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.159805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:47173 (local address 127.17.69.67:41203)
0812 06:52:11.160071 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.160077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.160700 (+ 623us) server_negotiation.cc:407] Connection header received
0812 06:52:11.160858 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.160862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.160924 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.160995 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.161627 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.162308 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.164117 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.164906 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.165045 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.165117 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.165391 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.165509 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 06:52:11.165828 32364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.159890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40187 (local address 127.17.69.65:52571)
0812 06:52:11.160690 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.160707 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.160833 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.161142 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.161146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.161184 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.161531 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.161539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.162533 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.162538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.163969 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.163980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.165063 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.165070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.165201 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.165224 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.165303 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.165416 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":24,"thread_start_us":425,"threads_started":1}
I20260812 06:52:11.165877 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.159645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.17.69.65:47173)
0812 06:52:11.160597 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.160611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.160723 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.161032 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.161037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.161062 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.161458 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.161472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.162447 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.162452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.163971 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.163983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.165127 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.165142 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.165252 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.165274 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.165351 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.165466 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":2,"thread_start_us":637,"threads_started":1}
I20260812 06:52:11.166697 32222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758d645562cc401ebcc1dfc4969848f8" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c" is_pre_election: true
I20260812 06:52:11.166699 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758d645562cc401ebcc1dfc4969848f8" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" is_pre_election: true
I20260812 06:52:11.167096 32222 raft_consensus.cc:2454] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 0.
I20260812 06:52:11.167107 31975 raft_consensus.cc:2454] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 0.
I20260812 06:52:11.167649 31591 leader_election.cc:302] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd11fcdeaa8d478a9e1f720bf1e0c3c6, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:11.167973 32361 raft_consensus.cc:2781] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:11.168125 32361 raft_consensus.cc:482] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:11.168220 32361 raft_consensus.cc:3037] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.170154 32361 raft_consensus.cc:504] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.170663 32361 leader_election.cc:288] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 election: Requested vote from peers 42658cf367ee485da2770938e977934c (127.17.69.67:41203), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:11.171016 32222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758d645562cc401ebcc1dfc4969848f8" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c"
I20260812 06:52:11.171072 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758d645562cc401ebcc1dfc4969848f8" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
I20260812 06:52:11.171172 32222 raft_consensus.cc:3037] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.171227 31975 raft_consensus.cc:3037] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.173174 32222 raft_consensus.cc:2454] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 1.
I20260812 06:52:11.173187 31975 raft_consensus.cc:2454] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 1.
I20260812 06:52:11.173520 31591 leader_election.cc:302] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, bd11fcdeaa8d478a9e1f720bf1e0c3c6, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:11.173789 32361 raft_consensus.cc:2781] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:11.174188 32361 raft_consensus.cc:686] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 LEADER]: Becoming Leader. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Running, Role: LEADER
I20260812 06:52:11.174633 32361 consensus_queue.cc:237] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } }
I20260812 06:52:11.178524 31428 catalog_manager.cc:5495] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65). New cstate: current_term: 1 leader_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } health_report { overall_health: UNKNOWN } } }
I20260812 06:52:11.213897 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.208553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.17.69.65:40477)
0812 06:52:11.208751 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.208755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.208806 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:52:11.209003 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.209006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.209110 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.209233 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:11.209596 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.210616 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.211890 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.212308 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.212417 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.212840 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.213331 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.213455 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.213568 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.213651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 06:52:11.219300 32307 tablet.cc:2369] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:52:11.221469 32222 raft_consensus.cc:1261] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Refusing update from remote peer bd11fcdeaa8d478a9e1f720bf1e0c3c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.221465 31975 raft_consensus.cc:1261] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Refusing update from remote peer bd11fcdeaa8d478a9e1f720bf1e0c3c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.222456 32369 consensus_queue.cc:1035] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:11.222795 32361 consensus_queue.cc:1035] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:52:11.226926 32072 tablet.cc:2369] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:52:11.236074 32377 mvcc.cc:205] Tried to move back new op lower bound from 7317575807874125824 to 7317575807694274560. Current Snapshot: MvccSnapshot[applied={T|T < 7317575807874125824}]
I20260812 06:52:11.246167 31428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:11.246430 31428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:11.249617 31428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Sending DeleteTablet for 3 replicas of tablet 758d645562cc401ebcc1dfc4969848f8
I20260812 06:52:11.250635 32186 tablet_service.cc:1514] Processing DeleteTablet for tablet 758d645562cc401ebcc1dfc4969848f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50826
I20260812 06:52:11.250820 31682 tablet_service.cc:1514] Processing DeleteTablet for tablet 758d645562cc401ebcc1dfc4969848f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33584
I20260812 06:52:11.251441 31937 tablet_service.cc:1514] Processing DeleteTablet for tablet 758d645562cc401ebcc1dfc4969848f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:47028
I20260812 06:52:11.252336 31428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:11.253100 31428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:11.255949 32400 tablet_replica.cc:330] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: stopping tablet replica
I20260812 06:52:11.256582 32399 tablet_replica.cc:330] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: stopping tablet replica
I20260812 06:52:11.257023 32400 raft_consensus.cc:2229] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:11.257557 32399 raft_consensus.cc:2229] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:11.258075 32400 raft_consensus.cc:2258] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.258747 32399 raft_consensus.cc:2258] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.259649 32396 tablet_replica.cc:330] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: stopping tablet replica
I20260812 06:52:11.259920 32396 raft_consensus.cc:2229] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:11.260329 32396 raft_consensus.cc:2258] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.263067 32396 ts_tablet_manager.cc:1905] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.264055 31937 tablet_service.cc:1467] Processing CreateTablet for tablet 11421ed945de444093f1e4c1add3bc60 (DEFAULT_TABLE table=test-table [id=fb9557ae447444989ea5445167965b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.264479 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11421ed945de444093f1e4c1add3bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.265141 32399 ts_tablet_manager.cc:1905] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.266973 32400 ts_tablet_manager.cc:1905] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.269688 32396 ts_tablet_manager.cc:1918] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:11.269824 32396 log.cc:1196] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/758d645562cc401ebcc1dfc4969848f8
I20260812 06:52:11.270272 32396 ts_tablet_manager.cc:1939] T 758d645562cc401ebcc1dfc4969848f8 P 42658cf367ee485da2770938e977934c: Deleting consensus metadata
I20260812 06:52:11.270839 31682 tablet_service.cc:1467] Processing CreateTablet for tablet 11421ed945de444093f1e4c1add3bc60 (DEFAULT_TABLE table=test-table [id=fb9557ae447444989ea5445167965b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.271330 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11421ed945de444093f1e4c1add3bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.271986 31412 catalog_manager.cc:4841] TS 42658cf367ee485da2770938e977934c (127.17.69.67:41203): tablet 758d645562cc401ebcc1dfc4969848f8 (table test-table [id=077d3c19e49a4510b9191f9c45a04c90]) successfully deleted
I20260812 06:52:11.272682 32352 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Bootstrap starting.
I20260812 06:52:11.274286 32352 tablet_bootstrap.cc:654] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.278224 32351 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Bootstrap starting.
I20260812 06:52:11.279224 32186 tablet_service.cc:1467] Processing CreateTablet for tablet 11421ed945de444093f1e4c1add3bc60 (DEFAULT_TABLE table=test-table [id=fb9557ae447444989ea5445167965b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:11.279624 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11421ed945de444093f1e4c1add3bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.279980 32351 tablet_bootstrap.cc:654] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.284554 32399 ts_tablet_manager.cc:1918] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:11.284968 32400 ts_tablet_manager.cc:1918] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:11.285185 32400 log.cc:1196] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/758d645562cc401ebcc1dfc4969848f8
I20260812 06:52:11.285732 32350 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Bootstrap starting.
I20260812 06:52:11.285750 32400 ts_tablet_manager.cc:1939] T 758d645562cc401ebcc1dfc4969848f8 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting consensus metadata
I20260812 06:52:11.286474 32399 log.cc:1196] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/758d645562cc401ebcc1dfc4969848f8
I20260812 06:52:11.287385 32350 tablet_bootstrap.cc:654] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.288015 32399 ts_tablet_manager.cc:1939] T 758d645562cc401ebcc1dfc4969848f8 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting consensus metadata
I20260812 06:52:11.289671 31411 catalog_manager.cc:4841] TS dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187): tablet 758d645562cc401ebcc1dfc4969848f8 (table test-table [id=077d3c19e49a4510b9191f9c45a04c90]) successfully deleted
I20260812 06:52:11.291194 31411 catalog_manager.cc:4841] TS bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477): tablet 758d645562cc401ebcc1dfc4969848f8 (table test-table [id=077d3c19e49a4510b9191f9c45a04c90]) successfully deleted
I20260812 06:52:11.296159 32350 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: No bootstrap required, opened a new log
I20260812 06:52:11.296291 32350 ts_tablet_manager.cc:1397] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260812 06:52:11.296942 32350 raft_consensus.cc:348] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.297211 32350 raft_consensus.cc:374] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.297345 32350 raft_consensus.cc:729] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42658cf367ee485da2770938e977934c, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.297422 32352 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: No bootstrap required, opened a new log
I20260812 06:52:11.297600 32352 ts_tablet_manager.cc:1397] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.006s
I20260812 06:52:11.297708 32351 tablet_bootstrap.cc:492] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: No bootstrap required, opened a new log
I20260812 06:52:11.297641 32350 consensus_queue.cc:260] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.297902 32351 ts_tablet_manager.cc:1397] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.005s
I20260812 06:52:11.298442 32352 raft_consensus.cc:348] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.298681 32352 raft_consensus.cc:374] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.298792 32352 raft_consensus.cc:729] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.298763 32351 raft_consensus.cc:348] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.299106 32351 raft_consensus.cc:374] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.299057 32352 consensus_queue.cc:260] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.299234 32351 raft_consensus.cc:729] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.299629 32352 ts_tablet_manager.cc:1428] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:11.299543 32351 consensus_queue.cc:260] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.300691 32351 ts_tablet_manager.cc:1428] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:52:11.302517 32350 ts_tablet_manager.cc:1428] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260812 06:52:11.382632 32362 raft_consensus.cc:482] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:11.382856 32362 raft_consensus.cc:504] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.384433 32362 leader_election.cc:288] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), 42658cf367ee485da2770938e977934c (127.17.69.67:41203)
I20260812 06:52:11.391677 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.385281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37295 (local address 127.17.69.67:41203)
0812 06:52:11.385452 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.385458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.385477 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:52:11.385721 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.385729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.385814 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.385876 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.386789 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.387748 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.389917 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.390761 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.390911 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.390991 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.391289 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.391415 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:52:11.391716 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.384722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.17.69.66:37295)
0812 06:52:11.385349 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.385370 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.385521 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.386103 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.386109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.386139 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.386570 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.386582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.387886 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.387890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.389771 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.389783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.390990 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.390997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.391143 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.391166 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.391250 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.391373 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":1718,"threads_started":1}
I20260812 06:52:11.392366 32222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11421ed945de444093f1e4c1add3bc60" candidate_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c" is_pre_election: true
I20260812 06:52:11.392611 32222 raft_consensus.cc:2454] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbd7b672b58f4e33ad7a103e17eb57e2 in term 0.
I20260812 06:52:11.393217 31858 leader_election.cc:302] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:11.393636 32362 raft_consensus.cc:2781] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:11.393778 32362 raft_consensus.cc:482] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:11.393872 32362 raft_consensus.cc:3037] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.394614 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.384899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:39571 (local address 127.17.69.65:40477)
0812 06:52:11.385136 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.385143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.387122 (+ 1979us) server_negotiation.cc:407] Connection header received
0812 06:52:11.387314 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.387319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.387394 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.387483 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.388063 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.389197 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.391636 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.392905 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.393035 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.393114 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.394144 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.394286 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:52:11.395289 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.384749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40477 (local address 127.17.69.66:39571)
0812 06:52:11.387007 (+ 2258us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.387025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.387176 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.387505 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.387510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.387535 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.387877 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.387890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.389327 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.389332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.391435 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.391451 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.393731 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.393737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.393941 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.393974 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.394815 (+ 841us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.394958 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2146,"thread_start_us":1978,"threads_started":1}
I20260812 06:52:11.396111 31724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11421ed945de444093f1e4c1add3bc60" candidate_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" is_pre_election: true
I20260812 06:52:11.396531 31724 raft_consensus.cc:2454] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbd7b672b58f4e33ad7a103e17eb57e2 in term 0.
I20260812 06:52:11.397022 32362 raft_consensus.cc:504] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.398042 31724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11421ed945de444093f1e4c1add3bc60" candidate_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
I20260812 06:52:11.398242 31724 raft_consensus.cc:3037] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.398537 32222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11421ed945de444093f1e4c1add3bc60" candidate_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c"
I20260812 06:52:11.398725 32222 raft_consensus.cc:3037] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.401420 31724 raft_consensus.cc:2454] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbd7b672b58f4e33ad7a103e17eb57e2 in term 1.
I20260812 06:52:11.401611 32222 raft_consensus.cc:2454] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbd7b672b58f4e33ad7a103e17eb57e2 in term 1.
I20260812 06:52:11.402022 31858 leader_election.cc:302] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:11.402426 32362 leader_election.cc:288] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [CANDIDATE]: Term 1 election: Requested vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), 42658cf367ee485da2770938e977934c (127.17.69.67:41203)
I20260812 06:52:11.402665 32362 raft_consensus.cc:2781] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:11.402833 32362 raft_consensus.cc:686] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 LEADER]: Becoming Leader. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Running, Role: LEADER
I20260812 06:52:11.403225 32362 consensus_queue.cc:237] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.416630 31428 catalog_manager.cc:5495] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } health_report { overall_health: UNKNOWN } } }
W20260812 06:52:11.489176 31819 tablet.cc:2369] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:52:11 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517528, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.126@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:11.502508 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.484594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36505 (local address 127.0.0.1:42920)
0812 06:52:11.485233 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.485253 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.485391 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.485682 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.485687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.486374 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:11.486660 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.486671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.487644 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.487648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.488615 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.488637 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.489701 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.489707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.489875 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.494920 (+ 5045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:11.494939 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:11.497065 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:11.499851 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:11.499925 (+ 74us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:11.499934 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:11.500383 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:11.500710 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:11.500715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:11.500720 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:11.500868 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:11.501461 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:11.501469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:11.501646 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.501786 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.502051 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":375,"threads_started":1}
I20260812 06:52:11.502529 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.484562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.17.69.126:36505)
0812 06:52:11.484811 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.484819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.485333 (+ 514us) server_negotiation.cc:407] Connection header received
0812 06:52:11.485496 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.485502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.485558 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.485634 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:11.486794 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.487510 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.488768 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.489311 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.489406 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.497245 (+ 7839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:11.497251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:11.497254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:11.497284 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:11.499724 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:11.500489 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:11.500495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:11.500498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:11.500613 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:11.500986 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:11.500992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:11.500994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:11.501338 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:11.501486 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.501897 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.502189 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":122}
I20260812 06:52:11.507128 31428 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:42920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 06:52:11.508119 31428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:11.515723 31682 tablet_service.cc:1467] Processing CreateTablet for tablet deb0a28721f34781b04a6a9168e25033 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb79878bc72f41258640ec99b15f5ab7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:52:11.516359 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb0a28721f34781b04a6a9168e25033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.518887 32186 tablet_service.cc:1467] Processing CreateTablet for tablet deb0a28721f34781b04a6a9168e25033 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb79878bc72f41258640ec99b15f5ab7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:52:11.519294 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb0a28721f34781b04a6a9168e25033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.524689 31937 tablet_service.cc:1467] Processing CreateTablet for tablet deb0a28721f34781b04a6a9168e25033 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb79878bc72f41258640ec99b15f5ab7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 06:52:11.525528 32350 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: Bootstrap starting.
I20260812 06:52:11.525547 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb0a28721f34781b04a6a9168e25033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:11.528393 32350 tablet_bootstrap.cc:654] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.535550 32350 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: No bootstrap required, opened a new log
I20260812 06:52:11.535722 32350 ts_tablet_manager.cc:1397] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260812 06:52:11.536345 32350 raft_consensus.cc:348] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.536554 32350 raft_consensus.cc:374] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.536639 32350 raft_consensus.cc:729] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42658cf367ee485da2770938e977934c, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.536882 32350 consensus_queue.cc:260] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.537292 32359 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.537654 32359 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.538134 32350 ts_tablet_manager.cc:1428] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 06:52:11.539499 32351 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Bootstrap starting.
I20260812 06:52:11.541383 32351 tablet_bootstrap.cc:654] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.543033 32352 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: Bootstrap starting.
I20260812 06:52:11.543998 32351 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: No bootstrap required, opened a new log
I20260812 06:52:11.544245 32351 ts_tablet_manager.cc:1397] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 06:52:11.544734 32352 tablet_bootstrap.cc:654] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:11.545127 32351 raft_consensus.cc:348] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.545480 32351 raft_consensus.cc:374] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.545640 32351 raft_consensus.cc:729] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.546010 32351 consensus_queue.cc:260] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.546542 32369 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.546892 32369 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.547036 32352 tablet_bootstrap.cc:492] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: No bootstrap required, opened a new log
I20260812 06:52:11.547195 32352 ts_tablet_manager.cc:1397] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260812 06:52:11.547928 32351 ts_tablet_manager.cc:1428] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 06:52:11.548151 32352 raft_consensus.cc:348] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.548393 32352 raft_consensus.cc:374] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:11.548475 32352 raft_consensus.cc:729] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Initialized, Role: FOLLOWER
I20260812 06:52:11.548909 32352 consensus_queue.cc:260] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.549335 32362 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.549585 32362 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.550869 32352 ts_tablet_manager.cc:1428] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 06:52:11.677505 32359 raft_consensus.cc:482] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:11.677739 32359 raft_consensus.cc:504] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.679183 32359 leader_election.cc:288] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:11.685415 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.679525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:34337 (local address 127.17.69.65:40477)
0812 06:52:11.679683 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.679688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.680242 (+ 554us) server_negotiation.cc:407] Connection header received
0812 06:52:11.680459 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.680464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.680544 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.680654 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.681355 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.682092 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.683793 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.684528 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.684682 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.684767 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.685022 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.685157 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:52:11.685468 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.679522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40477 (local address 127.17.69.67:34337)
0812 06:52:11.680154 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.680173 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.680312 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.680801 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.680807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.680834 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.681209 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.681220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.682224 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.682229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.683639 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.683651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.684738 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.684744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.684878 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.684902 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.684982 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.685113 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":338,"threads_started":1}
I20260812 06:52:11.685949 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.680314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40187 (local address 127.17.69.67:52345)
0812 06:52:11.680860 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.680929 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.681048 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.681378 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.681383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.681409 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:11.681731 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.681740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.682818 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.682822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.684146 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.684159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.685295 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.685302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.685429 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.685451 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.685541 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.685669 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":263,"threads_started":1}
I20260812 06:52:11.686138 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.680416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:52345 (local address 127.17.69.66:40187)
0812 06:52:11.681111 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.681115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.681131 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:52:11.681187 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.681191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.681246 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.681334 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:11.681892 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.682690 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.684298 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.685059 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.685223 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.685300 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.685722 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.685863 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":492,"threads_started":1}
I20260812 06:52:11.686139 31724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb0a28721f34781b04a6a9168e25033" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" is_pre_election: true
I20260812 06:52:11.686411 31724 raft_consensus.cc:2454] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 0.
I20260812 06:52:11.686695 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb0a28721f34781b04a6a9168e25033" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" is_pre_election: true
I20260812 06:52:11.686853 32111 leader_election.cc:302] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, bd11fcdeaa8d478a9e1f720bf1e0c3c6; no voters:
I20260812 06:52:11.686950 31975 raft_consensus.cc:2454] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 0.
I20260812 06:52:11.687249 32359 raft_consensus.cc:2781] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:11.687382 32359 raft_consensus.cc:482] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:11.687479 32359 raft_consensus.cc:3037] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.689707 32359 raft_consensus.cc:504] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.690280 32359 leader_election.cc:288] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 election: Requested vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:11.690711 31724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb0a28721f34781b04a6a9168e25033" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
I20260812 06:52:11.690800 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb0a28721f34781b04a6a9168e25033" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
I20260812 06:52:11.690904 31724 raft_consensus.cc:3037] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.690994 31975 raft_consensus.cc:3037] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:11.692790 31724 raft_consensus.cc:2454] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 1.
I20260812 06:52:11.692888 31975 raft_consensus.cc:2454] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 1.
I20260812 06:52:11.693140 32111 leader_election.cc:302] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, bd11fcdeaa8d478a9e1f720bf1e0c3c6; no voters:
I20260812 06:52:11.693378 32359 raft_consensus.cc:2781] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:11.693761 32359 raft_consensus.cc:686] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [term 1 LEADER]: Becoming Leader. State: Replica: 42658cf367ee485da2770938e977934c, State: Running, Role: LEADER
I20260812 06:52:11.694130 32359 consensus_queue.cc:237] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:11.696018 32441 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: TxnStatusTablet state changed. Reason: New leader 42658cf367ee485da2770938e977934c. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.696408 32441 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:52:11.697085 32447 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 06:52:11.697275 32447 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 06:52:11.697607 31428 catalog_manager.cc:5495] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c reported cstate change: term changed from 0 to 1, leader changed from <none> to 42658cf367ee485da2770938e977934c (127.17.69.67). New cstate: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } health_report { overall_health: HEALTHY } } }
I20260812 06:52:11.747095 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.740809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.17.69.67:41203)
0812 06:52:11.740976 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.740982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.741308 (+ 326us) server_negotiation.cc:407] Connection header received
0812 06:52:11.741575 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.741580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.741690 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.741824 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:11.742275 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.743111 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.744992 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.745403 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.745491 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.745995 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.746565 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.746665 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.746800 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.746876 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:52:11.747244 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.740910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.0.0.1:50840)
0812 06:52:11.741191 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.741213 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.741376 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.741859 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.741865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.741879 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:52:11.742125 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.742139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.743252 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.743258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.744851 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.744868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.745546 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.745551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.745736 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.745808 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.746671 (+ 863us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.746693 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.746786 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.746941 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:52:11.751708 31975 raft_consensus.cc:1261] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Refusing update from remote peer 42658cf367ee485da2770938e977934c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.751961 31724 raft_consensus.cc:1261] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Refusing update from remote peer 42658cf367ee485da2770938e977934c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.752348 32441 consensus_queue.cc:1035] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:11.752760 32359 consensus_queue.cc:1035] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:11.763643 32362 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: TxnStatusTablet state changed. Reason: New leader 42658cf367ee485da2770938e977934c. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.764035 32362 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.766557 32369 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: TxnStatusTablet state changed. Reason: New leader 42658cf367ee485da2770938e977934c. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.766844 32369 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.777710 32359 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.778002 32359 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:52:11.778750 32359 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.778993 32359 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P 42658cf367ee485da2770938e977934c: This TxnStatusTablet replica's current role is: LEADER
I20260812 06:52:11.785544 32369 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.785815 32369 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.787055 32362 tablet_replica.cc:439] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } } }
I20260812 06:52:11.787339 32362 tablet_replica.cc:442] T deb0a28721f34781b04a6a9168e25033 P dbd7b672b58f4e33ad7a103e17eb57e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 06:52:11.797825 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.791271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.17.69.66:40187)
0812 06:52:11.791469 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.791475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.791994 (+ 519us) server_negotiation.cc:407] Connection header received
0812 06:52:11.792210 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.792214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.792306 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.792421 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:11.792898 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.793564 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.795564 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.796069 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.796170 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.796655 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.797275 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.797381 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.797503 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.797569 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:52:11.812541 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.806880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.17.69.66:60099)
0812 06:52:11.807387 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.807402 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.807508 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.807809 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.807812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.807820 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:52:11.808057 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.808065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.809371 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.809375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.810546 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.810559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.811552 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.811556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.811662 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.811716 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.812116 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.812123 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.812173 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.812278 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260812 06:52:11.812605 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.807000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:60099 (local address 127.17.69.67:41203)
0812 06:52:11.807212 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.807218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.807473 (+ 255us) server_negotiation.cc:407] Connection header received
0812 06:52:11.807613 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.807619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.807707 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.807806 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:11.808192 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.809255 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.810682 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.811402 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.811532 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.811866 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.812007 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.812116 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.812220 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.812301 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:52:11.821985 31724 raft_consensus.cc:1261] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Refusing update from remote peer dbd7b672b58f4e33ad7a103e17eb57e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.821976 32222 raft_consensus.cc:1261] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Refusing update from remote peer dbd7b672b58f4e33ad7a103e17eb57e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:11.822671 32362 consensus_queue.cc:1035] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:11.822952 32411 consensus_queue.cc:1035] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:11.830958 32452 mvcc.cc:205] Tried to move back new op lower bound from 7317575810337472512 to 7317575808631922688. Current Snapshot: MvccSnapshot[applied={T|T < 7317575810337472512}]
I20260812 06:52:11.858646 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.852416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:39109 (local address 127.17.69.66:40187)
0812 06:52:11.852596 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:11.852603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:11.853166 (+ 563us) server_negotiation.cc:407] Connection header received
0812 06:52:11.853355 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:11.853361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:11.853440 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:11.853518 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:11.854064 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.855136 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.856661 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.857279 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.857390 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.857828 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.857958 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.858049 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:11.858250 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 06:52:11.858335 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:52:11.859431 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:11.852338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:40187 (local address 127.17.69.67:39109)
0812 06:52:11.853035 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:11.853055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:11.853229 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:11.853552 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:11.853557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:11.853569 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:52:11.853914 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.853923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.855253 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.855259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.856529 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:11.856546 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:11.857441 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:11.857448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:11.857601 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:11.857668 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:52:11.858083 (+ 415us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:52:11.858103 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 06:52:11.858879 (+ 776us) client_negotiation.cc:240] Negotiation successful
0812 06:52:11.859019 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":305,"threads_started":1}
I20260812 06:52:11.891115 31428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:11.891355 31428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:11.894421 31428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Sending DeleteTablet for 3 replicas of tablet 11421ed945de444093f1e4c1add3bc60
I20260812 06:52:11.895372 31682 tablet_service.cc:1514] Processing DeleteTablet for tablet 11421ed945de444093f1e4c1add3bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33584
I20260812 06:52:11.895385 32186 tablet_service.cc:1514] Processing DeleteTablet for tablet 11421ed945de444093f1e4c1add3bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50826
I20260812 06:52:11.895753 31937 tablet_service.cc:1514] Processing DeleteTablet for tablet 11421ed945de444093f1e4c1add3bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:11 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:47028
I20260812 06:52:11.930658 32487 tablet_replica.cc:330] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: stopping tablet replica
I20260812 06:52:11.931027 32487 raft_consensus.cc:2229] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:11.931713 32487 raft_consensus.cc:2258] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.932178 32485 tablet_replica.cc:330] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: stopping tablet replica
I20260812 06:52:11.933146 32485 raft_consensus.cc:2229] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:11.933494 32485 raft_consensus.cc:2258] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.933998 32487 ts_tablet_manager.cc:1905] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.937687 32485 ts_tablet_manager.cc:1905] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.948474 32485 ts_tablet_manager.cc:1918] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:11.955195 32487 ts_tablet_manager.cc:1918] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:11.958072 32486 tablet_replica.cc:330] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: stopping tablet replica
I20260812 06:52:11.958482 32487 log.cc:1196] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/11421ed945de444093f1e4c1add3bc60
I20260812 06:52:11.958796 32486 raft_consensus.cc:2229] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:11.958977 32487 ts_tablet_manager.cc:1939] T 11421ed945de444093f1e4c1add3bc60 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting consensus metadata
I20260812 06:52:11.960008 32485 log.cc:1196] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/11421ed945de444093f1e4c1add3bc60
I20260812 06:52:11.960237 31411 catalog_manager.cc:4841] TS dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187): tablet 11421ed945de444093f1e4c1add3bc60 (table test-table [id=fb9557ae447444989ea5445167965b46]) successfully deleted
I20260812 06:52:11.960453 32485 ts_tablet_manager.cc:1939] T 11421ed945de444093f1e4c1add3bc60 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting consensus metadata
I20260812 06:52:11.960724 32486 raft_consensus.cc:2258] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:11.961470 31411 catalog_manager.cc:4841] TS bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477): tablet 11421ed945de444093f1e4c1add3bc60 (table test-table [id=fb9557ae447444989ea5445167965b46]) successfully deleted
I20260812 06:52:11.962201 32486 ts_tablet_manager.cc:1905] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:11.977563 32486 ts_tablet_manager.cc:1918] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:11.977706 32486 log.cc:1196] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/11421ed945de444093f1e4c1add3bc60
I20260812 06:52:11.978215 32486 ts_tablet_manager.cc:1939] T 11421ed945de444093f1e4c1add3bc60 P 42658cf367ee485da2770938e977934c: Deleting consensus metadata
I20260812 06:52:11.981351 31412 catalog_manager.cc:4841] TS 42658cf367ee485da2770938e977934c (127.17.69.67:41203): tablet 11421ed945de444093f1e4c1add3bc60 (table test-table [id=fb9557ae447444989ea5445167965b46]) successfully deleted
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260812 06:52:12.015060 17685 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.111s user 0.000s sys 0.020s
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:52:12.059857 32498 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:12.045614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.17.69.126:36505)
0812 06:52:12.046122 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.046129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.046857 (+ 728us) server_negotiation.cc:407] Connection header received
0812 06:52:12.047163 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.047170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.047260 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.047386 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:12.049135 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.050276 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.052141 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.052743 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.052882 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.059535 (+ 6653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42936: BlockingRecv error: recv got EOF from 127.0.0.1:42936 (error 108)
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":252,"threads_started":1}
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM
I20260812 06:52:12.087472 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.063408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.17.69.126:36505)
0812 06:52:12.063584 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.063589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.070295 (+ 6706us) server_negotiation.cc:407] Connection header received
0812 06:52:12.070459 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.070465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.070543 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.070671 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:12.072343 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.073400 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.075188 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.075820 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.075942 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.080875 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:12.080884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:12.080889 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:12.080930 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:12.084506 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.085278 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.085285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.085288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.085452 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.085892 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.085898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.085901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.086269 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:12.086571 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.086876 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.087124 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:52:12.091259 31428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:12.091984 31428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:12.100006 32186 tablet_service.cc:1467] Processing CreateTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 (DEFAULT_TABLE table=test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.100551 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3449518d85734a87aed4d2a8c94fefe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.102778 31682 tablet_service.cc:1467] Processing CreateTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 (DEFAULT_TABLE table=test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.102795 31937 tablet_service.cc:1467] Processing CreateTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 (DEFAULT_TABLE table=test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.103214 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3449518d85734a87aed4d2a8c94fefe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.103225 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3449518d85734a87aed4d2a8c94fefe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.125064 32514 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Bootstrap starting.
I20260812 06:52:12.127152 32514 tablet_bootstrap.cc:654] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.127722 32513 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Bootstrap starting.
I20260812 06:52:12.129499 32513 tablet_bootstrap.cc:654] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.130650 32512 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Bootstrap starting.
I20260812 06:52:12.133285 32512 tablet_bootstrap.cc:654] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.135322 32514 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: No bootstrap required, opened a new log
I20260812 06:52:12.135507 32514 ts_tablet_manager.cc:1397] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20260812 06:52:12.136258 32514 raft_consensus.cc:348] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.136575 32514 raft_consensus.cc:374] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.136703 32514 raft_consensus.cc:729] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.136981 32514 consensus_queue.cc:260] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.138159 32514 ts_tablet_manager.cc:1428] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260812 06:52:12.140161 32512 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: No bootstrap required, opened a new log
I20260812 06:52:12.140342 32512 ts_tablet_manager.cc:1397] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20260812 06:52:12.141175 32512 raft_consensus.cc:348] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.141453 32512 raft_consensus.cc:374] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.141573 32512 raft_consensus.cc:729] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.141896 32512 consensus_queue.cc:260] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.144568 32512 ts_tablet_manager.cc:1428] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 06:52:12.147522 32513 tablet_bootstrap.cc:492] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: No bootstrap required, opened a new log
I20260812 06:52:12.147667 32513 ts_tablet_manager.cc:1397] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.010s
I20260812 06:52:12.148433 32513 raft_consensus.cc:348] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.148687 32513 raft_consensus.cc:374] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.148792 32513 raft_consensus.cc:729] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42658cf367ee485da2770938e977934c, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.149032 32513 consensus_queue.cc:260] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.149576 32513 ts_tablet_manager.cc:1428] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 06:52:12.265252 32369 raft_consensus.cc:482] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:12.265451 32369 raft_consensus.cc:504] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.266119 32369 leader_election.cc:288] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187), 42658cf367ee485da2770938e977934c (127.17.69.67:41203)
I20260812 06:52:12.266815 31973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3449518d85734a87aed4d2a8c94fefe0" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" is_pre_election: true
I20260812 06:52:12.267186 31973 raft_consensus.cc:2454] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 0.
I20260812 06:52:12.267669 31591 leader_election.cc:302] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd11fcdeaa8d478a9e1f720bf1e0c3c6, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:12.267655 32220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3449518d85734a87aed4d2a8c94fefe0" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c" is_pre_election: true
I20260812 06:52:12.267926 32369 raft_consensus.cc:2781] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:12.267920 32220 raft_consensus.cc:2454] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 0.
I20260812 06:52:12.267998 32369 raft_consensus.cc:482] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:12.268065 32369 raft_consensus.cc:3037] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.270079 32369 raft_consensus.cc:504] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.270650 32369 leader_election.cc:288] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 election: Requested vote from peers dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187), 42658cf367ee485da2770938e977934c (127.17.69.67:41203)
I20260812 06:52:12.271134 31973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3449518d85734a87aed4d2a8c94fefe0" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
I20260812 06:52:12.271126 32220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3449518d85734a87aed4d2a8c94fefe0" candidate_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42658cf367ee485da2770938e977934c"
I20260812 06:52:12.271317 31973 raft_consensus.cc:3037] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.271332 32220 raft_consensus.cc:3037] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.273700 31973 raft_consensus.cc:2454] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 1.
I20260812 06:52:12.274060 31591 leader_election.cc:302] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd11fcdeaa8d478a9e1f720bf1e0c3c6, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:12.274302 32369 raft_consensus.cc:2781] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:12.274749 32369 raft_consensus.cc:686] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 LEADER]: Becoming Leader. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Running, Role: LEADER
I20260812 06:52:12.274817 32220 raft_consensus.cc:2454] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11fcdeaa8d478a9e1f720bf1e0c3c6 in term 1.
I20260812 06:52:12.275103 32369 consensus_queue.cc:237] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.277966 31428 catalog_manager.cc:5495] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65). New cstate: current_term: 1 leader_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } health_report { overall_health: UNKNOWN } } }
I20260812 06:52:12.312584 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.306480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.17.69.65:40477)
0812 06:52:12.307158 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.307165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.307188 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:52:12.307269 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.307274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.307363 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.307478 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:12.307828 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.309002 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.310663 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.311320 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.311460 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.311824 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:12.311966 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:12.312061 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.312157 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.312252 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":384,"threads_started":1}
I20260812 06:52:12.315284 31973 raft_consensus.cc:1261] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Refusing update from remote peer bd11fcdeaa8d478a9e1f720bf1e0c3c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:12.315270 32220 raft_consensus.cc:1261] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Refusing update from remote peer bd11fcdeaa8d478a9e1f720bf1e0c3c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:12.315865 32520 consensus_queue.cc:1035] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:12.316257 32369 consensus_queue.cc:1035] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:12.322963 32455 mvcc.cc:205] Tried to move back new op lower bound from 7317575812357439488 to 7317575812201762816. Current Snapshot: MvccSnapshot[applied={T|T < 7317575812357439488 or (T in {7317575812357439488})}]
I20260812 06:52:12.329342 31428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:12.329588 31428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:12.335997 31428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Sending DeleteTablet for 3 replicas of tablet 3449518d85734a87aed4d2a8c94fefe0
I20260812 06:52:12.337242 32185 tablet_service.cc:1514] Processing DeleteTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50826
I20260812 06:52:12.337541 32486 tablet_replica.cc:330] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: stopping tablet replica
I20260812 06:52:12.337705 31937 tablet_service.cc:1514] Processing DeleteTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:47028
I20260812 06:52:12.337772 32486 raft_consensus.cc:2229] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:12.337858 31682 tablet_service.cc:1514] Processing DeleteTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33584
I20260812 06:52:12.337972 32486 raft_consensus.cc:2258] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.338137 32485 tablet_replica.cc:330] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: stopping tablet replica
I20260812 06:52:12.338109 32186 tablet_service.cc:1514] Processing DeleteTablet for tablet 3449518d85734a87aed4d2a8c94fefe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50826
I20260812 06:52:12.338366 32485 raft_consensus.cc:2229] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:12.338534 32487 tablet_replica.cc:330] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: stopping tablet replica
I20260812 06:52:12.338672 32485 raft_consensus.cc:2258] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.338750 32487 raft_consensus.cc:2229] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:12.338960 32487 raft_consensus.cc:2258] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.339187 31429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:12.339846 31429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:12.340095 32485 ts_tablet_manager.cc:1905] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.342696 32487 ts_tablet_manager.cc:1905] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.343201 32486 ts_tablet_manager.cc:1905] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.347606 32486 ts_tablet_manager.cc:1918] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:12.347750 32486 log.cc:1196] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/3449518d85734a87aed4d2a8c94fefe0
I20260812 06:52:12.348142 32486 ts_tablet_manager.cc:1939] T 3449518d85734a87aed4d2a8c94fefe0 P 42658cf367ee485da2770938e977934c: Deleting consensus metadata
I20260812 06:52:12.351073 31937 tablet_service.cc:1467] Processing CreateTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 (DEFAULT_TABLE table=test-table [id=4580ef3aab54487198f604091feddbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.351505 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daeff7385cb54ab4b2c77e9f08cda583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.351583 31682 tablet_service.cc:1467] Processing CreateTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 (DEFAULT_TABLE table=test-table [id=4580ef3aab54487198f604091feddbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.351940 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daeff7385cb54ab4b2c77e9f08cda583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.351204 32186 tablet_service.cc:1467] Processing CreateTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 (DEFAULT_TABLE table=test-table [id=4580ef3aab54487198f604091feddbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:12.354920 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daeff7385cb54ab4b2c77e9f08cda583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:12.357605 32485 ts_tablet_manager.cc:1918] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:12.357834 32485 log.cc:1196] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/3449518d85734a87aed4d2a8c94fefe0
I20260812 06:52:12.358098 32512 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Bootstrap starting.
I20260812 06:52:12.358510 32485 ts_tablet_manager.cc:1939] T 3449518d85734a87aed4d2a8c94fefe0 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting consensus metadata
I20260812 06:52:12.359751 32512 tablet_bootstrap.cc:654] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.359711 31411 catalog_manager.cc:4841] TS bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477): tablet 3449518d85734a87aed4d2a8c94fefe0 (table test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]) successfully deleted
I20260812 06:52:12.361392 31412 catalog_manager.cc:4841] TS 42658cf367ee485da2770938e977934c (127.17.69.67:41203): tablet 3449518d85734a87aed4d2a8c94fefe0 (table test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]) successfully deleted
W20260812 06:52:12.361788 31412 catalog_manager.cc:4808] TS 42658cf367ee485da2770938e977934c (127.17.69.67:41203): delete failed for tablet 3449518d85734a87aed4d2a8c94fefe0 because the tablet was not found. No further retry: Not found: Tablet not found: 3449518d85734a87aed4d2a8c94fefe0
I20260812 06:52:12.362643 32512 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: No bootstrap required, opened a new log
I20260812 06:52:12.362862 32512 ts_tablet_manager.cc:1397] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 06:52:12.364006 32512 raft_consensus.cc:348] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.364326 32512 raft_consensus.cc:374] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.364465 32512 raft_consensus.cc:729] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd7b672b58f4e33ad7a103e17eb57e2, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.364761 32514 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Bootstrap starting.
I20260812 06:52:12.364786 32512 consensus_queue.cc:260] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.366282 32514 tablet_bootstrap.cc:654] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.367023 32513 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Bootstrap starting.
I20260812 06:52:12.368309 32487 ts_tablet_manager.cc:1918] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:12.368516 32487 log.cc:1196] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/3449518d85734a87aed4d2a8c94fefe0
I20260812 06:52:12.368587 32513 tablet_bootstrap.cc:654] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:12.369422 32487 ts_tablet_manager.cc:1939] T 3449518d85734a87aed4d2a8c94fefe0 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting consensus metadata
I20260812 06:52:12.370433 31411 catalog_manager.cc:4841] TS dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187): tablet 3449518d85734a87aed4d2a8c94fefe0 (table test-table [id=6f960fb551bd42359eb8ec9d8eb2f1d4]) successfully deleted
I20260812 06:52:12.371030 32512 ts_tablet_manager.cc:1428] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260812 06:52:12.375850 32513 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: No bootstrap required, opened a new log
I20260812 06:52:12.376066 32513 ts_tablet_manager.cc:1397] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260812 06:52:12.376894 32514 tablet_bootstrap.cc:492] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: No bootstrap required, opened a new log
I20260812 06:52:12.376849 32513 raft_consensus.cc:348] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.377063 32514 ts_tablet_manager.cc:1397] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260812 06:52:12.377105 32513 raft_consensus.cc:374] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.377192 32513 raft_consensus.cc:729] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42658cf367ee485da2770938e977934c, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.377475 32513 consensus_queue.cc:260] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.377838 32514 raft_consensus.cc:348] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.378082 32514 raft_consensus.cc:374] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:12.378165 32514 raft_consensus.cc:729] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11fcdeaa8d478a9e1f720bf1e0c3c6, State: Initialized, Role: FOLLOWER
I20260812 06:52:12.378415 32514 consensus_queue.cc:260] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.378705 32513 ts_tablet_manager.cc:1428] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 06:52:12.379570 32514 ts_tablet_manager.cc:1428] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:12.532953 32441 raft_consensus.cc:482] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:12.533165 32441 raft_consensus.cc:504] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.533701 32441 leader_election.cc:288] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:12.534307 31973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeff7385cb54ab4b2c77e9f08cda583" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" is_pre_election: true
I20260812 06:52:12.534329 31723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeff7385cb54ab4b2c77e9f08cda583" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" is_pre_election: true
I20260812 06:52:12.534539 31973 raft_consensus.cc:2454] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 0.
I20260812 06:52:12.534629 31723 raft_consensus.cc:2454] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 0.
I20260812 06:52:12.534967 32111 leader_election.cc:302] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, dbd7b672b58f4e33ad7a103e17eb57e2; no voters:
I20260812 06:52:12.535236 32441 raft_consensus.cc:2781] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:12.535328 32441 raft_consensus.cc:482] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:12.535420 32441 raft_consensus.cc:3037] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.537355 32441 raft_consensus.cc:504] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.537839 32441 leader_election.cc:288] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 election: Requested vote from peers bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477), dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187)
I20260812 06:52:12.538276 31723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeff7385cb54ab4b2c77e9f08cda583" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6"
I20260812 06:52:12.538372 31973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeff7385cb54ab4b2c77e9f08cda583" candidate_uuid: "42658cf367ee485da2770938e977934c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2"
I20260812 06:52:12.538476 31723 raft_consensus.cc:3037] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.538585 31973 raft_consensus.cc:3037] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:12.540805 31723 raft_consensus.cc:2454] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 1.
I20260812 06:52:12.540859 31973 raft_consensus.cc:2454] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42658cf367ee485da2770938e977934c in term 1.
I20260812 06:52:12.541198 32111 leader_election.cc:302] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42658cf367ee485da2770938e977934c, bd11fcdeaa8d478a9e1f720bf1e0c3c6; no voters:
I20260812 06:52:12.541435 32441 raft_consensus.cc:2781] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:12.541870 32441 raft_consensus.cc:686] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 1 LEADER]: Becoming Leader. State: Replica: 42658cf367ee485da2770938e977934c, State: Running, Role: LEADER
I20260812 06:52:12.542163 32441 consensus_queue.cc:237] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } }
I20260812 06:52:12.544436 31428 catalog_manager.cc:5495] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c reported cstate change: term changed from 0 to 1, leader changed from <none> to 42658cf367ee485da2770938e977934c (127.17.69.67). New cstate: current_term: 1 leader_uuid: "42658cf367ee485da2770938e977934c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42658cf367ee485da2770938e977934c" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41203 } health_report { overall_health: HEALTHY } } }
I20260812 06:52:12.600339 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.595543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.17.69.67:41203)
0812 06:52:12.595996 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.596001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.596014 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:52:12.596059 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.596062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.596119 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.596219 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:12.596589 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.597278 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.598646 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.599062 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.599183 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.599597 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:12.599731 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:12.599810 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.599956 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.600031 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":224,"threads_started":1}
I20260812 06:52:12.614004 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.609132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:58115 (local address 127.17.69.67:41203)
0812 06:52:12.609322 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.609328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.609345 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:52:12.609401 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.609405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.609476 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.609565 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:12.610072 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.611091 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.612498 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.612918 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.613022 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.613377 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:12.613470 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:12.613539 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.613655 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.613718 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:52:12.614065 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.608489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41203 (local address 127.17.69.67:58115)
0812 06:52:12.609053 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:12.609073 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:12.609206 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:12.609602 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:12.609608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:12.609625 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 06:52:12.609903 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:12.609916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.611245 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.611251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:12.612368 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:12.612381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.613070 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.613077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:12.613205 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.613236 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 06:52:12.613552 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 06:52:12.613562 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 06:52:12.613639 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:52:12.613762 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":290,"threads_started":1}
I20260812 06:52:12.621304 31973 raft_consensus.cc:1261] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Refusing update from remote peer 42658cf367ee485da2770938e977934c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:12.621352 31723 raft_consensus.cc:1261] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Refusing update from remote peer 42658cf367ee485da2770938e977934c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:12.621797 32547 consensus_queue.cc:1035] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbd7b672b58f4e33ad7a103e17eb57e2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 40187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:12.622160 32441 consensus_queue.cc:1035] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd11fcdeaa8d478a9e1f720bf1e0c3c6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:12.664299 31428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:12.664556 31428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:12.667544 31428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a64d23dae44bb0af089be27ed7adce: Sending DeleteTablet for 3 replicas of tablet daeff7385cb54ab4b2c77e9f08cda583
I20260812 06:52:12.668536 32186 tablet_service.cc:1514] Processing DeleteTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50826
I20260812 06:52:12.668586 31682 tablet_service.cc:1514] Processing DeleteTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33584
I20260812 06:52:12.668905 31937 tablet_service.cc:1514] Processing DeleteTablet for tablet daeff7385cb54ab4b2c77e9f08cda583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:12 UTC) from {username='oryx', principal='oryx/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:47028
I20260812 06:52:12.668972 32485 tablet_replica.cc:330] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: stopping tablet replica
I20260812 06:52:12.669143 32485 raft_consensus.cc:2229] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:12.669356 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 31553
I20260812 06:52:12.669353 32485 raft_consensus.cc:2258] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.669461 32487 tablet_replica.cc:330] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: stopping tablet replica
I20260812 06:52:12.669624 32486 tablet_replica.cc:330] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: stopping tablet replica
I20260812 06:52:12.669649 32487 raft_consensus.cc:2229] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:12.669812 32486 raft_consensus.cc:2229] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:12.669838 32487 raft_consensus.cc:2258] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.670284 32486 raft_consensus.cc:2258] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:12.671159 32485 ts_tablet_manager.cc:1905] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.671543 32486 ts_tablet_manager.cc:1905] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.677697 32485 ts_tablet_manager.cc:1918] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:12.677850 32485 log.cc:1196] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/daeff7385cb54ab4b2c77e9f08cda583
I20260812 06:52:12.678272 32485 ts_tablet_manager.cc:1939] T daeff7385cb54ab4b2c77e9f08cda583 P bd11fcdeaa8d478a9e1f720bf1e0c3c6: Deleting consensus metadata
I20260812 06:52:12.679361 32486 ts_tablet_manager.cc:1918] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:12.679353 31411 catalog_manager.cc:4841] TS bd11fcdeaa8d478a9e1f720bf1e0c3c6 (127.17.69.65:40477): tablet daeff7385cb54ab4b2c77e9f08cda583 (table test-table [id=4580ef3aab54487198f604091feddbe4]) successfully deleted
I20260812 06:52:12.679493 32486 log.cc:1196] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/daeff7385cb54ab4b2c77e9f08cda583
I20260812 06:52:12.679924 32486 ts_tablet_manager.cc:1939] T daeff7385cb54ab4b2c77e9f08cda583 P 42658cf367ee485da2770938e977934c: Deleting consensus metadata
I20260812 06:52:12.680997 31412 catalog_manager.cc:4841] TS 42658cf367ee485da2770938e977934c (127.17.69.67:41203): tablet daeff7385cb54ab4b2c77e9f08cda583 (table test-table [id=4580ef3aab54487198f604091feddbe4]) successfully deleted
I20260812 06:52:12.681351 32487 ts_tablet_manager.cc:1905] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:12.685508 32487 ts_tablet_manager.cc:1918] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 06:52:12.685608 32487 log.cc:1196] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/daeff7385cb54ab4b2c77e9f08cda583
I20260812 06:52:12.686007 32487 ts_tablet_manager.cc:1939] T daeff7385cb54ab4b2c77e9f08cda583 P dbd7b672b58f4e33ad7a103e17eb57e2: Deleting consensus metadata
I20260812 06:52:12.686903 31411 catalog_manager.cc:4841] TS dbd7b672b58f4e33ad7a103e17eb57e2 (127.17.69.66:40187): tablet daeff7385cb54ab4b2c77e9f08cda583 (table test-table [id=4580ef3aab54487198f604091feddbe4]) successfully deleted
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.65@KRBTEST.COM
I20260812 06:52:12.700829 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.670480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.17.69.65:40477)
0812 06:52:12.670635 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.670641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.685702 (+ 15061us) server_negotiation.cc:407] Connection header received
0812 06:52:12.685771 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.685776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.685861 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.685964 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:12.687612 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.688278 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.690062 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.690543 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.690648 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.695405 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:12.695419 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:12.695435 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:12.695465 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:12.698059 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.698902 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.698908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.698911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.699032 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.699412 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.699415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.699417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.699790 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:12.700068 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.700436 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.700605 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:52:12.701536 31802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:12.821152 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 31553
I20260812 06:52:12.851203 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 31822
Aug 12 06:52:12 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.66@KRBTEST.COM
I20260812 06:52:12.865880 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:12.851914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.17.69.66:40187)
0812 06:52:12.852373 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:12.852378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:12.852392 (+ 14us) server_negotiation.cc:407] Connection header received
0812 06:52:12.852446 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:12.852449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:12.852507 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:12.852598 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:12.853800 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.854555 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.855755 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:12.856214 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:12.856324 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:12.860586 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:12.860599 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:12.860614 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:12.860651 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:12.863385 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.864131 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.864137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.864140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.864271 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:12.864619 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:12.864622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:12.864624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:12.864930 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:12.865179 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:12.865450 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 06:52:12.865615 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":237,"threads_started":1}
I20260812 06:52:12.866667 32055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:13.048457 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 31822
I20260812 06:52:13.075448 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 32077
Aug 12 06:52:13 dist-test-slave-6zbq krb5kdc[31361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.69.67@KRBTEST.COM
I20260812 06:52:13.089488 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:13.076311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.17.69.67:41203)
0812 06:52:13.076468 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:13.076473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:13.076490 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:52:13.076548 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:13.076552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:13.076610 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:13.076698 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:13.077878 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:13.078839 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:13.080097 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:13.080531 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:13.080622 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:13.084799 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:13.084812 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:13.084828 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:13.084862 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:13.087384 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:13.087944 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:13.087950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:13.087952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:13.088080 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:13.088357 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:13.088360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:13.088362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:13.088698 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:13.088934 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:13.089087 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 06:52:13.089235 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:52:13.090097 32297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:13.232168 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 32077
I20260812 06:52:13.268146 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 31384
I20260812 06:52:13.269320 31501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:13.381951 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 31384
2026-08-12T06:52:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6846 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:52:13 dist-test-slave-6zbq krb5kdc[32571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:52:13 dist-test-slave-6zbq krb5kdc[32571](info): set up 2 sockets
Aug 12 06:52:13 dist-test-slave-6zbq krb5kdc[32571](info): commencing operation
krb5kdc: starting...
W20260812 06:52:14.623929 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:52:14 dist-test-slave-6zbq krb5kdc[32571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:52:14.875931 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36409
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:36409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:15.202538 32593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:15.202790 32593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:15.202881 32593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:15.209683 32593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:15.209769 32593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:15.209816 32593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:15.209870 32593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:15.209920 32593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:15.219007 32593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:36409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:15.220032 32593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:15.221184 32593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:15.231840 32600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:15.231876 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:15.231914 32601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:15.232579 32593 server_base.cc:1034] running on GCE node
I20260812 06:52:15.233276 32593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:15.234524 32593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:15.235724 32593 hybrid_clock.cc:648] HybridClock initialized: now 1786517535235675 us; error 38 us; skew 500 ppm
I20260812 06:52:15.236027 32593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T06:52:15Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1915 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:52:15 dist-test-slave-6zbq krb5kdc[32614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:52:15 dist-test-slave-6zbq krb5kdc[32614](info): set up 2 sockets
Aug 12 06:52:15 dist-test-slave-6zbq krb5kdc[32614](info): commencing operation
krb5kdc: starting...
W20260812 06:52:16.521643 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:52:16 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:52:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:52:16.738562 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:44917
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:17.046051 32638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:17.046334 32638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:17.046480 32638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:17.053364 32638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:17.053464 32638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:17.053541 32638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:17.053620 32638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:17.053682 32638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:17.062880 32638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:17.063989 32638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:17.065145 32638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:17.075825 32652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:17.075863 32658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:17.075896 32654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:17.077044 32638 server_base.cc:1034] running on GCE node
I20260812 06:52:17.077696 32638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:17.078959 32638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:17.080160 32638 hybrid_clock.cc:648] HybridClock initialized: now 1786517537080113 us; error 30 us; skew 500 ppm
I20260812 06:52:17.080453 32638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:17 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517537, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:17.088235 32638 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:52:17.089591 32638 webserver.cc:460] Webserver started at http://127.17.69.126:40375/ using document root <none> and password file <none>
I20260812 06:52:17.090197 32638 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:17.090318 32638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:17.090619 32638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:17.093343 32638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "238a95a100c2023fa3660427f2b58468"
server_key_iv: "56cc07b7ead0d5b97249d0c4914af4cf"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.094043 32638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "238a95a100c2023fa3660427f2b58468"
server_key_iv: "56cc07b7ead0d5b97249d0c4914af4cf"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.098906 32638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:52:17.101991 32672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:17.103161 32638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:52:17.103317 32638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "238a95a100c2023fa3660427f2b58468"
server_key_iv: "56cc07b7ead0d5b97249d0c4914af4cf"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.103490 32638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:17.117198 32638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:17.120573 32638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:17.120810 32638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:17.137228 32638 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:44917
I20260812 06:52:17.137269 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:44917 every 8 connection(s)
I20260812 06:52:17.138419 32638 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:17.141729 32761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:17.147038 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 32638
I20260812 06:52:17.147357 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:52:17.148198 17685 external_mini_cluster.cc:1428] Setting key 09a0bf8b2ae82815894c2e0dd89fae42
I20260812 06:52:17.152416 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Bootstrap starting.
I20260812 06:52:17.155723 32761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:17.156752 32761 log.cc:824] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:17.159482 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: No bootstrap required, opened a new log
Aug 12 06:52:17 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:17.169090 32761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:17.169478 32761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:17.169610 32761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eeb1daf77341fbb4d79b69a8ff9dd8, State: Initialized, Role: FOLLOWER
I20260812 06:52:17.170209 32761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:17.170575 32761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:17.170573 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:17.150089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.17.69.126:44917)
0812 06:52:17.150874 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:17.150884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:17.150921 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:52:17.152563 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:17.152584 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:17.152929 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:17.153197 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:17.155049 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.156307 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.157824 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.158464 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.158584 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:17.162882 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:17.162893 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:17.162914 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:17.162955 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:17.166648 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:17.167253 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:17.167262 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:17.167269 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:17.167468 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:17.167776 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:17.167781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:17.167785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:17.168433 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:17.168710 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:17.168964 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:52:17.169363 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":383,"threads_started":1}
I20260812 06:52:17.170753 32761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:17.170939 32761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:17.173070 32761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:17.173496 32761 leader_election.cc:302] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92eeb1daf77341fbb4d79b69a8ff9dd8; no voters:
I20260812 06:52:17.174015 32761 leader_election.cc:288] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:52:17.174114 300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:17.175185 300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 LEADER]: Becoming Leader. State: Replica: 92eeb1daf77341fbb4d79b69a8ff9dd8, State: Running, Role: LEADER
I20260812 06:52:17.175644 300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:17.175880 32761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:17.178071 303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92eeb1daf77341fbb4d79b69a8ff9dd8. Latest consensus state: current_term: 1 leader_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } } }
I20260812 06:52:17.178536 303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:17.178542 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } } }
I20260812 06:52:17.178869 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:17.180498 318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:17.184641 318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:17.191780 318 catalog_manager.cc:1347] Generated new cluster ID: 40fa79181e25455491e2d22bf6a6bf7a
I20260812 06:52:17.191885 318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:17.216450 318 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:52:17.217584 318 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:17.227368 318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Generated new TSK 0
I20260812 06:52:17.227995 318 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:52:17.287202 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:17.600590 341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:17.600835 341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:17.600960 341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:17.607913 341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:17.608004 341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:17.608146 341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:17.617134 341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:17.618201 341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:17.619356 341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:17.631065 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:17.631098 359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:17.631289 357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:17.632138 341 server_base.cc:1034] running on GCE node
I20260812 06:52:17.632706 341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:17.633523 341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:17.634724 341 hybrid_clock.cc:648] HybridClock initialized: now 1786517537634677 us; error 39 us; skew 500 ppm
I20260812 06:52:17.635001 341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:17 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517537, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:17.637862 341 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:52:17.639202 341 webserver.cc:460] Webserver started at http://127.17.69.65:39341/ using document root <none> and password file <none>
I20260812 06:52:17.639799 341 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:17.639916 341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:17.640182 341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:17.642871 341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "c0734d4ca8e345b9bdc3485e31096136"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "0a5630b38ff7489aac95becd30698d51"
server_key_iv: "a73fb5bab202442fdfbe9c9d1ad2fe2e"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.643596 341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "c0734d4ca8e345b9bdc3485e31096136"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "0a5630b38ff7489aac95becd30698d51"
server_key_iv: "a73fb5bab202442fdfbe9c9d1ad2fe2e"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.648270 341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 06:52:17.651393 374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:17.652514 341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:52:17.652666 341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "c0734d4ca8e345b9bdc3485e31096136"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "0a5630b38ff7489aac95becd30698d51"
server_key_iv: "a73fb5bab202442fdfbe9c9d1ad2fe2e"
server_key_version: "encryptionkey@0"
I20260812 06:52:17.652868 341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:17.668458 341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:17.672331 341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:17.672569 341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:17.673419 341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:17.674644 341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:17.674723 341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:17.674841 341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:17.674898 341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:17.704473 341 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:34055
I20260812 06:52:17.704516 572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:34055 every 8 connection(s)
I20260812 06:52:17.705590 341 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:17.707067 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 341
I20260812 06:52:17.707325 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:52:17.708035 17685 external_mini_cluster.cc:1428] Setting key 207c1a99a5dd62b086bf94e71a43a77b
Aug 12 06:52:17 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517537, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:17.724361 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:17.708129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:58143 (local address 127.17.69.126:44917)
0812 06:52:17.708583 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:17.708589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:17.709591 (+ 1002us) server_negotiation.cc:407] Connection header received
0812 06:52:17.711493 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:17.711500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:17.711576 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:17.711680 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:17.713002 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.713923 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.715381 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.715904 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.716019 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:17.719038 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:17.719044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:17.719049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:17.719084 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:17.721581 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:17.722329 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:17.722332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:17.722334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:17.722476 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:17.722897 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:17.722900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:17.722902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:17.723146 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:17.723256 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:17.723962 (+ 706us) server_negotiation.cc:299] Negotiation successful
0812 06:52:17.724113 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260812 06:52:17.725545 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:17.708649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:58143)
0812 06:52:17.709422 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:17.709472 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:17.711243 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:17.711866 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:17.711873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:17.712327 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:17.712840 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:17.712857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.714046 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.714050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:17.715245 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:17.715258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:17.716074 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:17.716078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:17.716256 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:17.716827 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:17.716848 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:17.718839 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:17.721717 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:17.721725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:17.721738 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:17.722231 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:17.722569 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:17.722573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:17.722576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:17.722807 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:17.723260 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:17.723267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:17.723611 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 06:52:17.723918 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 06:52:17.724370 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":288,"threads_started":1}
I20260812 06:52:17.726965 576 heartbeater.cc:347] Connected to a master server at 127.17.69.126:44917
I20260812 06:52:17.727241 576 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:17.727985 576 heartbeater.cc:511] Master 127.17.69.126:44917 requested a full tablet report, sending...
I20260812 06:52:17.730005 32708 ts_manager.cc:194] Registered new tserver with Master: c0734d4ca8e345b9bdc3485e31096136 (127.17.69.65:34055)
I20260812 06:52:17.731382 32708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:58143
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:52:17.762507 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:18.071707 590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:18.071959 590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:18.072080 590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:18.078938 590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:18.079041 590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:18.079192 590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:18.088232 590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:18.089310 590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:18.090485 590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:18.102187 600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:18.102236 604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:18.102283 598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:18.103391 590 server_base.cc:1034] running on GCE node
I20260812 06:52:18.103905 590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:18.104712 590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:18.105911 590 hybrid_clock.cc:648] HybridClock initialized: now 1786517538105874 us; error 25 us; skew 500 ppm
I20260812 06:52:18.106176 590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:18.109027 590 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:52:18.110301 590 webserver.cc:460] Webserver started at http://127.17.69.66:35171/ using document root <none> and password file <none>
I20260812 06:52:18.110913 590 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:18.111032 590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:18.111322 590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:18.113943 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "1f5fcf89ee0b414c85b6c4442f58231b"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "be33fa5b799872fc7632dbc77db14998"
server_key_iv: "b493a680032123e35329ea5bf1bcbf62"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.114706 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "1f5fcf89ee0b414c85b6c4442f58231b"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "be33fa5b799872fc7632dbc77db14998"
server_key_iv: "b493a680032123e35329ea5bf1bcbf62"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.119333 590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:52:18.122423 619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.123566 590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:52:18.123689 590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "1f5fcf89ee0b414c85b6c4442f58231b"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "be33fa5b799872fc7632dbc77db14998"
server_key_iv: "b493a680032123e35329ea5bf1bcbf62"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.123840 590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:18.150456 590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:18.154326 590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:18.154615 590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:18.155443 590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:18.156694 590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:18.156764 590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.156867 590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:18.156914 590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.186257 590 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:42431
I20260812 06:52:18.186314 832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:42431 every 8 connection(s)
I20260812 06:52:18.187381 590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:18.191200 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 590
I20260812 06:52:18.191377 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:52:18.191759 17685 external_mini_cluster.cc:1428] Setting key 9419d07153b258d65c18f1ed579b63b2
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:18.205480 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.189564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:41551 (local address 127.17.69.126:44917)
0812 06:52:18.189726 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:18.189730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:18.190799 (+ 1069us) server_negotiation.cc:407] Connection header received
0812 06:52:18.192647 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:18.192653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:18.192746 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:18.192834 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:18.194330 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.195083 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.196561 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.196960 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.197066 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.200259 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:18.200264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:18.200267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:18.200297 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:18.202778 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.203421 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.203426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.203428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.203554 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.203941 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.203946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.203949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.204193 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:18.204327 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:18.204956 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 06:52:18.205179 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:52:18.206278 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.190012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:41551)
0812 06:52:18.190661 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:18.190695 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:18.192376 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:18.193059 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:18.193066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:18.193653 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:18.194167 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:18.194182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.195223 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.195227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:18.196419 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:18.196431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.197167 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.197174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:18.197395 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.197962 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:18.197983 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:18.200062 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:18.202900 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:18.202908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:18.202922 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:18.203313 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:18.203654 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:18.203659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:18.203663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:18.203836 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:18.204317 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:18.204324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:18.204604 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 06:52:18.204826 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 06:52:18.205277 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":271,"threads_started":1}
I20260812 06:52:18.207576 833 heartbeater.cc:347] Connected to a master server at 127.17.69.126:44917
I20260812 06:52:18.207882 833 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:18.208714 833 heartbeater.cc:511] Master 127.17.69.126:44917 requested a full tablet report, sending...
I20260812 06:52:18.210145 32708 ts_manager.cc:194] Registered new tserver with Master: 1f5fcf89ee0b414c85b6c4442f58231b (127.17.69.66:42431)
I20260812 06:52:18.211203 32708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:41551
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:52:18.245218 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:18.557143 843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:18.557406 843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:18.557534 843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:18.564412 843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:18.564507 843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:18.564661 843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:18.573637 843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:18.574692 843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:18.575848 843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:18.587464 854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:18.587509 852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:18.587539 857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:18.588439 843 server_base.cc:1034] running on GCE node
I20260812 06:52:18.588980 843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:18.589781 843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:18.590966 843 hybrid_clock.cc:648] HybridClock initialized: now 1786517538590918 us; error 40 us; skew 500 ppm
I20260812 06:52:18.591217 843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:18.593959 843 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:52:18.595196 843 webserver.cc:460] Webserver started at http://127.17.69.67:33263/ using document root <none> and password file <none>
I20260812 06:52:18.595774 843 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:18.595867 843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:18.596097 843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:18.598685 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "7f2dc279723f449fbd11822d1549d040"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "8f7b2f816251a6ff63ab1f23fdda4207"
server_key_iv: "1781d0c5b2c4e960124c58ce2b5335a8"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.599380 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "7f2dc279723f449fbd11822d1549d040"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "8f7b2f816251a6ff63ab1f23fdda4207"
server_key_iv: "1781d0c5b2c4e960124c58ce2b5335a8"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.604257 843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 06:52:18.607641 866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.608996 843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:18.609141 843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "7f2dc279723f449fbd11822d1549d040"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "8f7b2f816251a6ff63ab1f23fdda4207"
server_key_iv: "1781d0c5b2c4e960124c58ce2b5335a8"
server_key_version: "encryptionkey@0"
I20260812 06:52:18.609345 843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:18.627604 843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:18.631479 843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:18.631704 843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:18.632507 843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:18.633737 843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:18.633806 843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.633894 843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:18.633935 843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:18.661638 843 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:43517
I20260812 06:52:18.661675 1050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:43517 every 8 connection(s)
I20260812 06:52:18.662803 843 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:18.664731 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 843
I20260812 06:52:18.664938 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:18.665524 17685 external_mini_cluster.cc:1428] Setting key a55105ab487b8cd549813509d7f0682d
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517538, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:18.681603 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.665177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:36565 (local address 127.17.69.126:44917)
0812 06:52:18.665352 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:18.665357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:18.666754 (+ 1397us) server_negotiation.cc:407] Connection header received
0812 06:52:18.668831 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:18.668836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:18.668901 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:18.668981 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:18.670283 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.671033 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.672451 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.672861 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.672954 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.676007 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:18.676013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:18.676017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:18.676047 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:18.678529 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.679282 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.679288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.679290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.679406 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.679873 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.679876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.679878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.680206 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:18.680341 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:18.681024 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 06:52:18.681276 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:52:18.682435 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.665754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:36565)
0812 06:52:18.666587 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:18.666642 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:18.668564 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:18.669171 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:18.669178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:18.669641 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:18.670158 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:18.670174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.671157 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.671161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:18.672322 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:18.672333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.673023 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.673028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:18.673226 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.673830 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:18.673855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:18.675819 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:18.678651 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:18.678657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:18.678666 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:18.679170 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:18.679545 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:18.679549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:18.679552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:18.679767 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:18.680328 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:18.680336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:18.680660 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 06:52:18.680932 (+ 272us) client_negotiation.cc:240] Negotiation successful
0812 06:52:18.681386 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":335,"threads_started":1}
I20260812 06:52:18.683596 1051 heartbeater.cc:347] Connected to a master server at 127.17.69.126:44917
I20260812 06:52:18.683825 1051 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:18.684409 1051 heartbeater.cc:511] Master 127.17.69.126:44917 requested a full tablet report, sending...
I20260812 06:52:18.685644 32708 ts_manager.cc:194] Registered new tserver with Master: 7f2dc279723f449fbd11822d1549d040 (127.17.69.67:43517)
I20260812 06:52:18.686416 32708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:36565
I20260812 06:52:18.692595 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.65@KRBTEST.COM" created.
I20260812 06:52:18.734659 576 heartbeater.cc:503] Master 127.17.69.126:44917 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
Entry for principal oryx/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab.
WARNING: no policy specified for oryx/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
Entry for principal oryx/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab.
WARNING: no policy specified for oryx/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
Entry for principal oryx/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab.
I20260812 06:52:18.832715 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 341
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:52:18.852629 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.833890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.17.69.65:34055)
0812 06:52:18.834558 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:18.834565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:18.834601 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:52:18.834666 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:18.834672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:18.834877 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:18.834999 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:18.836743 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.837758 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.839685 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.840270 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.840369 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.845186 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:18.845201 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:18.845218 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:18.845258 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:18.848798 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.849585 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.849604 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.849608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.849756 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.850200 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.850205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.850208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.850663 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:18.851819 (+ 1156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:18.851976 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 06:52:18.852228 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":344,"threads_started":1}
I20260812 06:52:18.854187 567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:18.939293 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 341
I20260812 06:52:18.962852 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 590
Aug 12 06:52:18 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:52:18.977200 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:18.963743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.17.69.66:42431)
0812 06:52:18.964271 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:18.964278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:18.964304 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:52:18.964362 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:18.964368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:18.964562 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:18.964690 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:18.965823 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.966859 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.968001 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:18.968511 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:18.968611 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:18.972408 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:18.972420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:18.972431 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:18.972462 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:18.974918 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.975447 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.975453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.975455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.975569 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:18.975834 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:18.975837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:18.975838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:18.976163 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:18.976387 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:18.976631 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 06:52:18.976823 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":250,"threads_started":1}
I20260812 06:52:18.978502 828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:19.057909 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 590
I20260812 06:52:19.080583 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 843
Aug 12 06:52:19 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:52:19.095983 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:19.081674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.17.69.67:43517)
0812 06:52:19.082322 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:19.082329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:19.082382 (+ 53us) server_negotiation.cc:407] Connection header received
0812 06:52:19.082451 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:19.082458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:19.082658 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:19.082792 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:19.084042 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:19.085095 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:19.086457 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:19.087023 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:19.087120 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:19.091185 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:19.091197 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:19.091210 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:19.091248 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:19.093751 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:19.094314 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:19.094320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:19.094323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:19.094478 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:19.094763 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:19.094766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:19.094768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:19.095078 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:19.095299 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:19.095495 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 06:52:19.095695 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":334,"threads_started":1}
I20260812 06:52:19.097206 1045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:19.176622 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 843
I20260812 06:52:19.197890 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 32638
I20260812 06:52:19.198961 32755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:19.299115 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 32638
I20260812 06:52:19.326377 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:44917
--webserver_interface=127.17.69.126
--webserver_port=40375
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:19.638522 1074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:19.638783 1074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:19.638913 1074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:19.645823 1074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:19.645924 1074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:19.645998 1074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:19.646075 1074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:19.646142 1074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:19.655215 1074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=40375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:19.656304 1074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:19.657428 1074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:19.668226 1083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:19.668315 1084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:19.668725 1088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:19.669028 1074 server_base.cc:1034] running on GCE node
I20260812 06:52:19.669689 1074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:19.670967 1074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:19.672163 1074 hybrid_clock.cc:648] HybridClock initialized: now 1786517539672121 us; error 29 us; skew 500 ppm
I20260812 06:52:19.672452 1074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:19 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517539, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:19.675300 1074 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:52:19.676621 1074 webserver.cc:460] Webserver started at http://127.17.69.126:40375/ using document root <none> and password file <none>
I20260812 06:52:19.677202 1074 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:19.677309 1074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:19.682154 1074 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:19.684675 1097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:19.685809 1074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:52:19.685941 1074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "238a95a100c2023fa3660427f2b58468"
server_key_iv: "56cc07b7ead0d5b97249d0c4914af4cf"
server_key_version: "encryptionkey@0"
I20260812 06:52:19.686870 1074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:19.699229 1074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:19.703912 1074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:19.704304 1074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:19.723642 1074 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:44917
I20260812 06:52:19.723649 1174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:44917 every 8 connection(s)
I20260812 06:52:19.724931 1074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:19.729341 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 1074
I20260812 06:52:19.730291 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:34055
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:44917
--webserver_port=39341
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:52:19.732092 1176 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:52:19.736018 1176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Bootstrap starting.
I20260812 06:52:19.754941 1176 log.cc:824] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:19.763311 1176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:52:19.763824 1176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Bootstrap complete.
I20260812 06:52:19.772208 1176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:19.772959 1176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eeb1daf77341fbb4d79b69a8ff9dd8, State: Initialized, Role: FOLLOWER
I20260812 06:52:19.773555 1176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:19.773763 1176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:19.773871 1176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:19.774024 1176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:52:19.777508 1176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:19.777940 1176 leader_election.cc:302] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92eeb1daf77341fbb4d79b69a8ff9dd8; no voters:
I20260812 06:52:19.778487 1176 leader_election.cc:288] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:52:19.778609 1182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:52:19.779693 1182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [term 2 LEADER]: Becoming Leader. State: Replica: 92eeb1daf77341fbb4d79b69a8ff9dd8, State: Running, Role: LEADER
I20260812 06:52:19.780267 1182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } }
I20260812 06:52:19.780483 1176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:19.782770 1196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92eeb1daf77341fbb4d79b69a8ff9dd8. Latest consensus state: current_term: 2 leader_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } } }
I20260812 06:52:19.783149 1196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:19.783666 1184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92eeb1daf77341fbb4d79b69a8ff9dd8" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 44917 } } }
I20260812 06:52:19.783978 1184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:19.797068 1201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:19.802871 1201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:19.804281 1201 catalog_manager.cc:1259] Loaded cluster ID: 40fa79181e25455491e2d22bf6a6bf7a
I20260812 06:52:19.804452 1201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:19.808403 1201 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:19.809862 1201 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 92eeb1daf77341fbb4d79b69a8ff9dd8: Loaded TSK: 0
I20260812 06:52:19.811407 1201 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:52:20.057386 1178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:20.057646 1178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:20.057775 1178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:20.065311 1178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:20.065433 1178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:20.065603 1178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:20.077888 1178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.65.keytab
--principal=oryx/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:34055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=39341
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:20.079190 1178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:20.080389 1178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:20.094403 1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:20.094412 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:20.094542 1241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:20.095891 1178 server_base.cc:1034] running on GCE node
I20260812 06:52:20.096614 1178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:20.097828 1178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:20.099071 1178 hybrid_clock.cc:648] HybridClock initialized: now 1786517540099013 us; error 45 us; skew 500 ppm
I20260812 06:52:20.099414 1178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517540, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:20.103291 1178 init.cc:377] Logged in from keytab as oryx/127.17.69.65@KRBTEST.COM (short username oryx)
I20260812 06:52:20.104947 1178 webserver.cc:460] Webserver started at http://127.17.69.65:39341/ using document root <none> and password file <none>
I20260812 06:52:20.105793 1178 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:20.105945 1178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:20.112466 1178 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:52:20.115456 1261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.116923 1178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:20.117084 1178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "c0734d4ca8e345b9bdc3485e31096136"
format_stamp: "Formatted at 2026-08-12 06:52:17 on dist-test-slave-6zbq"
server_key: "0a5630b38ff7489aac95becd30698d51"
server_key_iv: "a73fb5bab202442fdfbe9c9d1ad2fe2e"
server_key_version: "encryptionkey@0"
I20260812 06:52:20.118356 1178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:20.139521 1178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:20.144579 1178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:20.144850 1178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:20.145923 1178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:20.147533 1178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:20.147647 1178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.147745 1178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:20.147816 1178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.177995 1178 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:34055
I20260812 06:52:20.178062 1470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:34055 every 8 connection(s)
I20260812 06:52:20.179488 1178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:20.187520 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 1178
I20260812 06:52:20.188083 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:42431
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:44917
--webserver_port=35171
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:52:20.506824 1478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:20.507051 1478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:20.507166 1478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:20.513988 1478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:20.514074 1478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:20.514219 1478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:20.523265 1478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.66.keytab
--principal=oryx/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:42431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=35171
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:20.524420 1478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:20.525583 1478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:20.538550 1490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:20.538528 1496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:20.539027 1493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:20.540235 1478 server_base.cc:1034] running on GCE node
I20260812 06:52:20.540779 1478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:20.541646 1478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:20.542836 1478 hybrid_clock.cc:648] HybridClock initialized: now 1786517540542782 us; error 54 us; skew 500 ppm
I20260812 06:52:20.543131 1478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517540, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:20.546263 1478 init.cc:377] Logged in from keytab as oryx/127.17.69.66@KRBTEST.COM (short username oryx)
I20260812 06:52:20.547643 1478 webserver.cc:460] Webserver started at http://127.17.69.66:35171/ using document root <none> and password file <none>
I20260812 06:52:20.548242 1478 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:20.548432 1478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:20.553874 1478 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:52:20.556583 1507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.557930 1478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:52:20.558069 1478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "1f5fcf89ee0b414c85b6c4442f58231b"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "be33fa5b799872fc7632dbc77db14998"
server_key_iv: "b493a680032123e35329ea5bf1bcbf62"
server_key_version: "encryptionkey@0"
I20260812 06:52:20.559067 1478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:20.574812 1478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:20.578300 1478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:20.578557 1478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:20.579418 1478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:20.580688 1478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:20.580786 1478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.580884 1478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:20.580957 1478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:20.629578 1478 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:42431
I20260812 06:52:20.629705 1705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:42431 every 8 connection(s)
I20260812 06:52:20.630955 1478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:20.635164 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 1478
I20260812 06:52:20.635756 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:43517
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:44917
--webserver_port=33263
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:20 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 06:52:20.960064 1718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:20.960327 1718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:20.960482 1718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:20.967976 1718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:20.968083 1718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:20.968247 1718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:20.977308 1718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/krb5kdc/oryx_127.17.69.67.keytab
--principal=oryx/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:43517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=33263
--tserver_master_addrs=127.17.69.126:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:20.978434 1718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:20.979578 1718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:20.994364 1733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:20.994401 1735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:21.001547 1737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:21.002377 1718 server_base.cc:1034] running on GCE node
I20260812 06:52:21.002948 1718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:21.006453 1718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:21.007704 1718 hybrid_clock.cc:648] HybridClock initialized: now 1786517541007643 us; error 50 us; skew 500 ppm
I20260812 06:52:21.007987 1718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517541, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:21.010954 1718 init.cc:377] Logged in from keytab as oryx/127.17.69.67@KRBTEST.COM (short username oryx)
I20260812 06:52:21.012545 1718 webserver.cc:460] Webserver started at http://127.17.69.67:33263/ using document root <none> and password file <none>
I20260812 06:52:21.013175 1718 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:21.013302 1718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:21.018214 1718 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:52:21.022686 1752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:21.024046 1718 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260812 06:52:21.024200 1718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "7f2dc279723f449fbd11822d1549d040"
format_stamp: "Formatted at 2026-08-12 06:52:18 on dist-test-slave-6zbq"
server_key: "8f7b2f816251a6ff63ab1f23fdda4207"
server_key_iv: "1781d0c5b2c4e960124c58ce2b5335a8"
server_key_version: "encryptionkey@0"
I20260812 06:52:21.025127 1718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:21.049414 1718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:21.053185 1718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:21.053417 1718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:21.054379 1718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:21.055683 1718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:21.055778 1718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:21.055881 1718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:21.055956 1718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:21.114176 1718 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:43517
I20260812 06:52:21.114230 1953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:43517 every 8 connection(s)
I20260812 06:52:21.115566 1718 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:21.121888 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 1718
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:21.156189 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.122797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.17.69.126:44917)
0812 06:52:21.130407 (+ 7610us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.130412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.133068 (+ 2656us) server_negotiation.cc:407] Connection header received
0812 06:52:21.133146 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.133152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.133257 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.133381 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.135707 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.137080 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.139160 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.139833 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.139963 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.144832 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:21.144844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:21.144865 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:21.144922 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:21.150433 (+ 5511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:21.151385 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:21.151396 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:21.151406 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:21.151705 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:21.152214 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:21.152220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:21.152224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:21.152902 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:21.153262 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:21.154637 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0812 06:52:21.155100 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7458,"thread_start_us":7314,"threads_started":1}
W20260812 06:52:21.446774 1474 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.254s user 0.000s sys 0.015s
W20260812 06:52:21.448534 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:20.182379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:32785 (local address 127.17.69.126:44917)
0812 06:52:20.183159 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:20.183170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:20.183775 (+ 605us) server_negotiation.cc:407] Connection header received
0812 06:52:20.185843 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:20.185855 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:20.186139 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:20.186456 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:20.187723 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.189057 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.190609 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.191222 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.191398 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.448170 (+1256772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:32785: BlockingRecv error: recv got EOF from 127.17.69.65:32785 (error 108)
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":350,"threads_started":1}
I20260812 06:52:21.449189 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:20.182489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:32785)
0812 06:52:20.183612 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:20.183646 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:20.185328 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:20.186555 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:20.186562 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:20.187049 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:20.187535 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:20.187552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.189215 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.189218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:20.190466 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:20.190479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.191451 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.191459 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:20.191701 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:20.192302 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:20.192324 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.447844 (+1255520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":734,"threads_started":1}
W20260812 06:52:21.450387 1471 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.69.126:44917: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:21.464238 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.451244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:47819)
0812 06:52:21.451559 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:21.451600 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:21.451736 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:21.452261 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:21.452269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:21.452839 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:21.453098 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.453113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.454684 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.454689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.456371 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.456388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.457751 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.457758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.457964 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.458754 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:21.458780 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.463916 (+ 5136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171}
W20260812 06:52:21.464447 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:21.451442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:47819 (local address 127.17.69.126:44917)
0812 06:52:21.451632 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.451639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.451661 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:52:21.451864 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.451871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.451960 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.452074 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.453254 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.454533 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.456573 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.457313 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.457415 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.464176 (+ 6761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:47819: BlockingRecv error: recv got EOF from 127.17.69.65:47819 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:21.476508 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.465276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:56069)
0812 06:52:21.465676 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:21.465695 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:21.465826 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:21.466185 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:21.466190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:21.466639 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:21.466869 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.466880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.468461 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.468467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.470083 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.470099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.471135 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.471141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.471332 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.471998 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:21.472021 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.476218 (+ 4197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280}
W20260812 06:52:21.476732 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:21.465357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:56069 (local address 127.17.69.126:44917)
0812 06:52:21.465509 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.465515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.465795 (+ 280us) server_negotiation.cc:407] Connection header received
0812 06:52:21.465931 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.465937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.466038 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.466144 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.467038 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.468265 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.470251 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.470987 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.471116 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.476444 (+ 5328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:56069: BlockingRecv error: recv got EOF from 127.17.69.65:56069 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 06:52:21.477007 1471 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:52:21.832525 1715 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.186s user 0.005s sys 0.016s
W20260812 06:52:21.833904 1713 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:20.633803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:39993 (local address 127.17.69.126:44917)
0812 06:52:20.634332 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:20.634360 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:20.635346 (+ 986us) server_negotiation.cc:407] Connection header received
0812 06:52:20.638189 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:20.638195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:20.638273 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:20.638404 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:20.640554 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.641550 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.644086 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.644807 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.644943 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.833527 (+1188584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:39993: BlockingRecv error: recv got EOF from 127.17.69.66:39993 (error 108)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":255,"threads_started":1}
I20260812 06:52:21.834115 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:20.634127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:39993)
0812 06:52:20.635083 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:20.635135 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:20.637846 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:20.638707 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:20.638720 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:20.639499 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:20.640348 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:20.640374 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.641709 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.641715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:20.643873 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:20.643890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:20.645298 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:20.645306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:20.645647 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:20.646446 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:20.646478 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.833238 (+1186760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":350,"threads_started":1}
W20260812 06:52:21.835234 1706 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.69.126:44917: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:21.846411 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.835936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:55757)
0812 06:52:21.836151 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:21.836173 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:21.836320 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:21.836605 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:21.836610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:21.837147 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:21.837389 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.837400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.838424 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.838430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.840080 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.840096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.840790 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.840797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.840992 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.841638 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:21.841663 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.846060 (+ 4397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:52:21.846426 1713 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:21.836033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:55757 (local address 127.17.69.126:44917)
0812 06:52:21.836188 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.836193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.836241 (+ 48us) server_negotiation.cc:407] Connection header received
0812 06:52:21.836421 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.836425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.836481 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.836555 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.837542 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.838279 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.840232 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.840618 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.840718 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.846209 (+ 5491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:55757: BlockingRecv error: recv got EOF from 127.17.69.66:55757 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:21 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:21.855096 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.847222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:37267)
0812 06:52:21.847371 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:21.847387 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:21.847460 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:21.847869 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:21.847874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:21.848152 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:21.848298 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.848307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.849281 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.849286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.850203 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.850214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.850870 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.850876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.851031 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.851561 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:21.851576 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:21.854857 (+ 3281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:52:21.855355 1713 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:21.847466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37267 (local address 127.17.69.126:44917)
0812 06:52:21.847622 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.847626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.847639 (+ 13us) server_negotiation.cc:407] Connection header received
0812 06:52:21.847675 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.847678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.847735 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.847817 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.848437 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.849141 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.850332 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.850731 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.850837 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.855085 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:37267: BlockingRecv error: recv got EOF from 127.17.69.66:37267 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 06:52:21.855422 1706 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:52:22.142522 1958 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.002s user 0.000s sys 0.019s
W20260812 06:52:22.143571 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:21.119625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:43391 (local address 127.17.69.126:44917)
0812 06:52:21.126697 (+ 7072us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:21.126704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:21.130374 (+ 3670us) server_negotiation.cc:407] Connection header received
0812 06:52:21.130456 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:21.130463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:21.130560 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:21.130680 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:21.133079 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.134517 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.137088 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.137776 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.137876 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.143383 (+1005507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:43391: BlockingRecv error: recv got EOF from 127.17.69.67:43391 (error 108)
Metrics: {"server-negotiator.queue_time_us":6943,"thread_start_us":6819,"threads_started":1}
I20260812 06:52:22.144059 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:21.118957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:43391)
0812 06:52:21.126906 (+ 7949us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:21.126959 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:21.129604 (+ 2645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:21.131065 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:21.131078 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:21.131958 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:21.132840 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.132868 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.134735 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.134742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.136892 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:21.136910 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:21.138250 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:21.138257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:21.138587 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:21.140309 (+ 1722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:21.140341 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:22.143206 (+1002865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7612,"thread_start_us":7457,"threads_started":1}
W20260812 06:52:22.144974 1955 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.69.126:44917: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:22.158105 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:22.145622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:50033)
0812 06:52:22.145798 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:22.145813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:22.145908 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:22.146152 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:22.146156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:22.146513 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:22.146685 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.146694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.147608 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.147611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.148549 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.148560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.149197 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.149201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.149317 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.154476 (+ 5159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:22.154494 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:22.157905 (+ 3411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 06:52:22.158299 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:22.145665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:50033 (local address 127.17.69.126:44917)
0812 06:52:22.145822 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:22.145826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:22.145856 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:52:22.145989 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:22.145992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:22.146048 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:22.146122 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:22.146790 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.147503 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.148653 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.149065 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.149147 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.158107 (+ 8960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:50033: BlockingRecv error: recv got EOF from 127.17.69.67:50033 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:22.166106 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:22.158729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:38821)
0812 06:52:22.158883 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:22.158898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:22.159000 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:22.159342 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:22.159347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:22.159646 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:22.159797 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.159805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.160771 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.160774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.161713 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.161724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.162546 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.162551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.162712 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.163228 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:22.163244 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:22.165933 (+ 2689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 06:52:22.166379 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:22.158801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:38821 (local address 127.17.69.126:44917)
0812 06:52:22.159046 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:22.159052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:22.159094 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:52:22.159139 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:22.159144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:22.159213 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:22.159292 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:22.159931 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.160637 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.161845 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.162406 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.162519 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.166122 (+ 3603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:38821: BlockingRecv error: recv got EOF from 127.17.69.67:38821 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
W20260812 06:52:22.166409 1955 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:22.486073 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:22.477602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:34693)
0812 06:52:22.478121 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:22.478136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:22.478252 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:22.478721 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:22.478727 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:22.479179 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:22.479342 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.479352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.480336 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.480342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.481336 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.481349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.482086 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.482092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.482266 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.482778 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:22.482795 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:22.485760 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":285,"threads_started":1}
W20260812 06:52:22.486186 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:22.477657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:34693 (local address 127.17.69.126:44917)
0812 06:52:22.477831 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:22.477838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:22.478218 (+ 380us) server_negotiation.cc:407] Connection header received
0812 06:52:22.478497 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:22.478503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:22.478584 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:22.478689 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:22.479489 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.480190 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.481475 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.481915 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.482022 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.485977 (+ 3955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:34693: BlockingRecv error: recv got EOF from 127.17.69.65:34693 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:22 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:22.864733 1971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:22.855986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:53485)
0812 06:52:22.856525 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:22.856541 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:22.856672 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:22.856973 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:22.856978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:22.857546 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:22.857745 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.857758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.858832 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.858837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.859793 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:22.859805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.860533 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.860539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:22.860700 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.861238 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:22.861256 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:22.864425 (+ 3169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":313,"threads_started":1}
W20260812 06:52:22.864878 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:22.856083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:53485 (local address 127.17.69.126:44917)
0812 06:52:22.856344 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:22.856351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:22.856621 (+ 270us) server_negotiation.cc:407] Connection header received
0812 06:52:22.856780 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:22.856786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:22.856849 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:22.856930 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:22.857882 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.858688 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.859931 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:22.860353 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:22.860463 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:22.864651 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:53485: BlockingRecv error: recv got EOF from 127.17.69.66:53485 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:23.175488 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:23.166922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:54477)
0812 06:52:23.167451 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:23.167465 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:23.167589 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:23.167908 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:23.167914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:23.168346 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:23.168493 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.168503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.169481 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.169485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.170595 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.170611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.171553 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.171557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.171680 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.172194 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:23.172211 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:23.175256 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":313,"threads_started":1}
W20260812 06:52:23.175645 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:23.167024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:54477 (local address 127.17.69.126:44917)
0812 06:52:23.167238 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:23.167244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:23.167548 (+ 304us) server_negotiation.cc:407] Connection header received
0812 06:52:23.167700 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:23.167706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:23.167786 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:23.167866 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:23.168639 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.169347 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.170738 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.171388 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.171482 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.175450 (+ 3968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:54477: BlockingRecv error: recv got EOF from 127.17.69.67:54477 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:23.495826 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:23.486977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:53267)
0812 06:52:23.487457 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:23.487473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:23.487591 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:23.487846 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:23.487849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:23.488253 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:23.488436 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.488446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.489367 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.489372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.490387 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.490404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.491110 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.491114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.491238 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.491778 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:23.491802 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:23.495567 (+ 3765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":264,"threads_started":1}
W20260812 06:52:23.495980 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:23.487035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:53267 (local address 127.17.69.126:44917)
0812 06:52:23.487283 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:23.487289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:23.487547 (+ 258us) server_negotiation.cc:407] Connection header received
0812 06:52:23.487678 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:23.487681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:23.487743 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:23.487819 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:23.488549 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.489241 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.490497 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.490965 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.491051 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.495763 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:53267: BlockingRecv error: recv got EOF from 127.17.69.65:53267 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:23 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:23.874194 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:23.865666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:41229)
0812 06:52:23.866144 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:23.866159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:23.866295 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:23.866582 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:23.866586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:23.866960 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:23.867141 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.867149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.868073 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.868076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.868969 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:23.868980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.869613 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.869620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:23.869794 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.870686 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:23.870704 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:23.873955 (+ 3251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
W20260812 06:52:23.874323 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:23.865706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:41229 (local address 127.17.69.126:44917)
0812 06:52:23.865884 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:23.865888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:23.866224 (+ 336us) server_negotiation.cc:407] Connection header received
0812 06:52:23.866412 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:23.866416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:23.866477 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:23.866553 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:23.867288 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.867972 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.869069 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:23.869482 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:23.869574 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:23.874117 (+ 4543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:41229: BlockingRecv error: recv got EOF from 127.17.69.66:41229 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:24.184760 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:24.176261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:55423)
0812 06:52:24.176732 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:24.176747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:24.176855 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:24.177131 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:24.177134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:24.177497 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:24.177715 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.177726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.178790 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.178794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.179696 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.179708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.180395 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.180401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.180562 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.181102 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:24.181120 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:24.184523 (+ 3403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":250,"threads_started":1}
W20260812 06:52:24.184927 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:24.176392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:55423 (local address 127.17.69.126:44917)
0812 06:52:24.176555 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:24.176559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:24.176810 (+ 251us) server_negotiation.cc:407] Connection header received
0812 06:52:24.176939 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:24.176943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:24.177015 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:24.177093 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:24.177876 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.178654 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.179834 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.180233 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.180338 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.184710 (+ 4372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:55423: BlockingRecv error: recv got EOF from 127.17.69.67:55423 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:24.504930 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:24.496608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:41753)
0812 06:52:24.497035 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:24.497051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:24.497167 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:24.497491 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:24.497495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:24.497843 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:24.498014 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.498023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.499041 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.499046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.499956 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.499970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.500634 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.500638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.500759 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.501283 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:24.501300 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:24.504696 (+ 3396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":215,"threads_started":1}
W20260812 06:52:24.505093 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:24.496653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:41753 (local address 127.17.69.126:44917)
0812 06:52:24.496866 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:24.496872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:24.497122 (+ 250us) server_negotiation.cc:407] Connection header received
0812 06:52:24.497299 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:24.497304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:24.497372 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:24.497444 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:24.498123 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.498904 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.500087 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.500499 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.500589 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.504891 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:41753: BlockingRecv error: recv got EOF from 127.17.69.65:41753 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:24 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:24.883194 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:24.875075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:60373)
0812 06:52:24.875514 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:24.875529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:24.875639 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:24.875879 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:24.875882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:24.876227 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:24.876393 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.876402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.877329 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.877332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.878228 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:24.878239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.878868 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.878872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:24.878988 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.879572 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:24.879591 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:24.882860 (+ 3269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":218,"threads_started":1}
W20260812 06:52:24.883384 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:24.875142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:60373 (local address 127.17.69.126:44917)
0812 06:52:24.875345 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:24.875351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:24.875598 (+ 247us) server_negotiation.cc:407] Connection header received
0812 06:52:24.875725 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:24.875729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:24.875789 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:24.875864 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:24.876500 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.877217 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.878325 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:24.878765 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:24.878848 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:24.883101 (+ 4253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:60373: BlockingRecv error: recv got EOF from 127.17.69.66:60373 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:25.193333 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:25.185543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:46139)
0812 06:52:25.185982 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:25.185997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:25.186103 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:25.186391 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:25.186394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:25.186738 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:25.186905 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.186913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.187856 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.187860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.188769 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.188781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.189369 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.189373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.189503 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.189994 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:25.190011 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:25.193105 (+ 3094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
W20260812 06:52:25.193493 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:25.185646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:46139 (local address 127.17.69.126:44917)
0812 06:52:25.185860 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:25.185866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:25.186066 (+ 200us) server_negotiation.cc:407] Connection header received
0812 06:52:25.186197 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:25.186201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:25.186284 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:25.186379 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:25.187011 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.187739 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.188873 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.189267 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.189352 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.193291 (+ 3939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:46139: BlockingRecv error: recv got EOF from 127.17.69.67:46139 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:25.514259 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:25.505828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:38111)
0812 06:52:25.506369 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:25.506386 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:25.506497 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:25.506804 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:25.506807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:25.507187 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:25.507363 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.507372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.508301 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.508304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.509283 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.509296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.509888 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.509892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.510009 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.510625 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:25.510644 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:25.514000 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":262,"threads_started":1}
W20260812 06:52:25.514441 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:25.505866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:38111 (local address 127.17.69.126:44917)
0812 06:52:25.506022 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:25.506026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:25.506450 (+ 424us) server_negotiation.cc:407] Connection header received
0812 06:52:25.506583 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:25.506586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:25.506647 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:25.506732 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:25.507469 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.508186 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.509400 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.509784 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.509871 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.514188 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:38111: BlockingRecv error: recv got EOF from 127.17.69.65:38111 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:25 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:25.892838 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:25.884113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:56365)
0812 06:52:25.884603 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:25.884619 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:25.884736 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:25.885040 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:25.885046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:25.885546 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:25.885797 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.885812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.886873 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.886879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.887892 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:25.887904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.888619 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.888625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:25.888801 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.889316 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:25.889332 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:25.892484 (+ 3152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":246,"threads_started":1}
W20260812 06:52:25.892899 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:25.884122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:56365 (local address 127.17.69.126:44917)
0812 06:52:25.884292 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:25.884297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:25.884684 (+ 387us) server_negotiation.cc:407] Connection header received
0812 06:52:25.884838 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:25.884842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:25.884898 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:25.884999 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:25.885946 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.886745 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.888027 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:25.888476 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:25.888582 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:25.892693 (+ 4111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:56365: BlockingRecv error: recv got EOF from 127.17.69.66:56365 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:26.202903 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:26.194175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:54097)
0812 06:52:26.194676 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:26.194692 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:26.194798 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:26.195078 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:26.195081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:26.195566 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:26.195784 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.195799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.197024 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.197028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.198086 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.198106 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.198722 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.198726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.198848 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.199346 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:26.199363 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:26.202658 (+ 3295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":269,"threads_started":1}
W20260812 06:52:26.203074 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:26.194220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:54097 (local address 127.17.69.126:44917)
0812 06:52:26.194409 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:26.194413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:26.194756 (+ 343us) server_negotiation.cc:407] Connection header received
0812 06:52:26.194883 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:26.194886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:26.194966 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:26.195044 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:26.195932 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.196905 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.198194 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.198624 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.198714 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.202838 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:54097: BlockingRecv error: recv got EOF from 127.17.69.67:54097 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:26.523877 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:26.515131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:53235)
0812 06:52:26.515648 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:26.515665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:26.515788 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:26.516103 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:26.516106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:26.516528 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:26.516696 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.516706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.517628 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.517632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.518592 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.518603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.519463 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.519468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.519591 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.520271 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:26.520293 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:26.523566 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":276,"threads_started":1}
W20260812 06:52:26.524017 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:26.515209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:53235 (local address 127.17.69.126:44917)
0812 06:52:26.515368 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:26.515372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:26.515725 (+ 353us) server_negotiation.cc:407] Connection header received
0812 06:52:26.515918 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:26.515923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:26.515989 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:26.516075 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:26.516811 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.517528 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.518746 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.519308 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.519430 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.523797 (+ 4367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:53235: BlockingRecv error: recv got EOF from 127.17.69.65:53235 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:26 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:26.902382 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:26.893757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:52721)
0812 06:52:26.894275 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:26.894290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:26.894442 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:26.894752 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:26.894757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:26.895212 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:26.895360 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.895369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.896351 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.896357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.897317 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:26.897334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.898031 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.898037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:26.898210 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.898756 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:26.898773 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:26.902049 (+ 3276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
W20260812 06:52:26.902518 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:26.893940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:52721 (local address 127.17.69.126:44917)
0812 06:52:26.894150 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:26.894157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:26.894383 (+ 226us) server_negotiation.cc:407] Connection header received
0812 06:52:26.894558 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:26.894564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:26.894629 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:26.894707 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:26.895494 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.896226 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.897447 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:26.897897 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:26.898004 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:26.902258 (+ 4254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:52721: BlockingRecv error: recv got EOF from 127.17.69.66:52721 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:27.216099 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:27.203692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:58921)
0812 06:52:27.204139 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:27.204157 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:27.204279 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:27.204583 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:27.204588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:27.205033 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:27.205181 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.205191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.206145 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.206151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.207087 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.207099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.207768 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.207774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.207933 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.212432 (+ 4499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:27.212450 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:27.215841 (+ 3391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
W20260812 06:52:27.216293 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:27.203756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:58921 (local address 127.17.69.126:44917)
0812 06:52:27.203930 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:27.203936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:27.204237 (+ 301us) server_negotiation.cc:407] Connection header received
0812 06:52:27.204390 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:27.204394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:27.204454 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:27.204533 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:27.205321 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.206011 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.207225 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.207633 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.207746 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.216082 (+ 8336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:58921: BlockingRecv error: recv got EOF from 127.17.69.67:58921 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:27.533533 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:27.524856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:60543)
0812 06:52:27.525396 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:27.525413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:27.525564 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:27.525880 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:27.525885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:27.526306 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:27.526506 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.526517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.527496 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.527500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.528486 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.528501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.529240 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.529246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.529413 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.529920 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:27.529936 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:27.533317 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":258,"threads_started":1}
W20260812 06:52:27.533751 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:27.524913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:60543 (local address 127.17.69.126:44917)
0812 06:52:27.525161 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:27.525168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:27.525520 (+ 352us) server_negotiation.cc:407] Connection header received
0812 06:52:27.525678 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:27.525682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:27.525744 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:27.525830 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:27.526644 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.527361 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.528620 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.529096 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.529200 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.533516 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:60543: BlockingRecv error: recv got EOF from 127.17.69.65:60543 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:27 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:27.911465 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:27.903595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:40251)
0812 06:52:27.904044 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:27.904064 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:27.904181 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:27.904458 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:27.904461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:27.904865 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:27.905044 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.905053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.905939 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.905942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.906878 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:27.906891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.907506 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.907510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:27.907631 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.908165 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:27.908181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:27.911243 (+ 3062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":230,"threads_started":1}
W20260812 06:52:27.911600 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:27.903667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:40251 (local address 127.17.69.126:44917)
0812 06:52:27.903849 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:27.903853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:27.904129 (+ 276us) server_negotiation.cc:407] Connection header received
0812 06:52:27.904266 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:27.904270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:27.904350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:27.904430 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:27.905150 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.905832 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.906981 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:27.907392 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:27.907478 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:27.911427 (+ 3949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:40251: BlockingRecv error: recv got EOF from 127.17.69.66:40251 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:28.225411 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:28.217146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:45463)
0812 06:52:28.217613 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:28.217629 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:28.217737 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:28.217988 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:28.217991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:28.218394 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:28.218567 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.218576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.219500 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.219503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.220525 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.220537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.221152 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.221156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.221283 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.221775 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:28.221793 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:28.225193 (+ 3400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":251,"threads_started":1}
W20260812 06:52:28.225564 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:28.217206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45463 (local address 127.17.69.126:44917)
0812 06:52:28.217398 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:28.217406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:28.217700 (+ 294us) server_negotiation.cc:407] Connection header received
0812 06:52:28.217821 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:28.217826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:28.217886 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:28.217963 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:28.218668 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.219400 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.220632 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.221012 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.221125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.225364 (+ 4239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:45463: BlockingRecv error: recv got EOF from 127.17.69.67:45463 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:28.542335 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:28.534329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:54371)
0812 06:52:28.534946 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:28.534961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:28.535074 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:28.535326 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:28.535329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:28.535714 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:28.535897 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.535907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.536820 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.536824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.537730 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.537741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.538391 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.538395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.538512 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.539023 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:28.539039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:28.542080 (+ 3041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":271,"threads_started":1}
W20260812 06:52:28.542496 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:28.534420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:54371 (local address 127.17.69.126:44917)
0812 06:52:28.534579 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:28.534584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:28.535024 (+ 440us) server_negotiation.cc:407] Connection header received
0812 06:52:28.535154 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:28.535157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:28.535217 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:28.535300 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:28.536003 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.536716 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.537850 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.538259 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.538365 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.542286 (+ 3921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:54371: BlockingRecv error: recv got EOF from 127.17.69.65:54371 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:28 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:28.920773 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:28.912293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:38997)
0812 06:52:28.912903 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:28.912918 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:28.913047 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:28.913404 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:28.913410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:28.913861 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:28.914039 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.914048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.914992 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.914996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.915945 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:28.915956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.916599 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.916603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:28.916716 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.917263 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:28.917281 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:28.920519 (+ 3238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":276,"threads_started":1}
W20260812 06:52:28.920940 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:28.912369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:38997 (local address 127.17.69.126:44917)
0812 06:52:28.912591 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:28.912598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:28.913005 (+ 407us) server_negotiation.cc:407] Connection header received
0812 06:52:28.913163 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:28.913168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:28.913247 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:28.913359 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:28.914163 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.914896 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.916051 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:28.916488 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:28.916576 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:28.920706 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:38997: BlockingRecv error: recv got EOF from 127.17.69.66:38997 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:29.234881 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:29.226231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:56653)
0812 06:52:29.226799 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:29.226814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:29.226922 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:29.227172 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:29.227175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:29.227570 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:29.227749 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.227758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.228713 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.228717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.229701 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.229713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.230445 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.230450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.230571 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.231151 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:29.231169 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:29.234537 (+ 3368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":240,"threads_started":1}
W20260812 06:52:29.234915 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:29.226308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:56653 (local address 127.17.69.126:44917)
0812 06:52:29.226481 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:29.226486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:29.226875 (+ 389us) server_negotiation.cc:407] Connection header received
0812 06:52:29.227007 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:29.227010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:29.227069 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:29.227147 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:29.227892 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.228600 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.229826 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.230233 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.230333 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.234716 (+ 4383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:56653: BlockingRecv error: recv got EOF from 127.17.69.67:56653 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:29.551965 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:29.543240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:49541)
0812 06:52:29.543866 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:29.543882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:29.543998 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:29.544261 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:29.544264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:29.544663 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:29.544853 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.544868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.546179 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.546183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.547156 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.547171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.547821 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.547826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.547938 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.548539 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:29.548558 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:29.551721 (+ 3163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":380,"threads_started":1}
W20260812 06:52:29.552111 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:29.543430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:49541 (local address 127.17.69.126:44917)
0812 06:52:29.543587 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:29.543592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:29.543960 (+ 368us) server_negotiation.cc:407] Connection header received
0812 06:52:29.544093 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:29.544097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:29.544160 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:29.544228 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:29.545006 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.546018 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.547267 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.547677 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.547766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.551892 (+ 4126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:49541: BlockingRecv error: recv got EOF from 127.17.69.65:49541 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:29 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:29.930460 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:29.921578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:55499)
0812 06:52:29.922065 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:29.922083 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:29.922204 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:29.922536 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:29.922539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:29.922995 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:29.923177 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.923187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.924397 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.924402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.925336 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:29.925349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.925951 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.925955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:29.926068 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.926751 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:29.926770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:29.930178 (+ 3408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
W20260812 06:52:29.930622 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:29.921635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:55499 (local address 127.17.69.126:44917)
0812 06:52:29.921798 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:29.921802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:29.922187 (+ 385us) server_negotiation.cc:407] Connection header received
0812 06:52:29.922295 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:29.922299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:29.922409 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:29.922485 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:29.923327 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.924261 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.925444 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:29.925824 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:29.925906 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:29.930427 (+ 4521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:55499: BlockingRecv error: recv got EOF from 127.17.69.66:55499 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:30.244202 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:30.235676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:36781)
0812 06:52:30.236139 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:30.236154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:30.236264 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:30.236512 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:30.236515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:30.236912 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:30.237089 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.237098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.238024 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.238027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.239255 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.239270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.240082 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.240086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.240211 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.240717 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:30.240734 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:30.243867 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":234,"threads_started":1}
W20260812 06:52:30.244321 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:30.235758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:36781 (local address 127.17.69.126:44917)
0812 06:52:30.235972 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:30.235978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:30.236221 (+ 243us) server_negotiation.cc:407] Connection header received
0812 06:52:30.236348 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:30.236352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:30.236413 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:30.236486 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:30.237200 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.237925 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.239392 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.239919 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.240035 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.244078 (+ 4043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:36781: BlockingRecv error: recv got EOF from 127.17.69.67:36781 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:30.561381 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:30.552813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:56335)
0812 06:52:30.553312 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:30.553327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:30.553457 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:30.553785 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:30.553790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:30.554221 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:30.554416 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.554429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.555426 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.555430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.556385 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.556399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.557100 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.557106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.557270 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.557782 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:30.557799 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:30.561149 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":258,"threads_started":1}
W20260812 06:52:30.561530 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:30.552876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:56335 (local address 127.17.69.126:44917)
0812 06:52:30.553065 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:30.553071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:30.553409 (+ 338us) server_negotiation.cc:407] Connection header received
0812 06:52:30.553575 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:30.553580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:30.553643 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:30.553734 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:30.554552 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.555286 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.556522 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.556944 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.557055 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.561325 (+ 4270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:56335: BlockingRecv error: recv got EOF from 127.17.69.65:56335 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:30 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:30.939980 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:30.931350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:58921)
0812 06:52:30.931863 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:30.931879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:30.932010 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:30.932328 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:30.932334 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:30.932761 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:30.932915 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.932925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.933954 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.933958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.934925 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:30.934938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.935639 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.935645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:30.935810 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.936371 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:30.936389 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:30.939759 (+ 3370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":270,"threads_started":1}
W20260812 06:52:30.940162 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:30.931404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58921 (local address 127.17.69.126:44917)
0812 06:52:30.931580 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:30.931587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:30.931968 (+ 381us) server_negotiation.cc:407] Connection header received
0812 06:52:30.932127 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:30.932131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:30.932194 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:30.932277 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:30.933078 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.933815 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.935079 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:30.935485 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:30.935595 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:30.939929 (+ 4334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:58921: BlockingRecv error: recv got EOF from 127.17.69.66:58921 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:31.253707 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:31.245140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:52431)
0812 06:52:31.245639 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:31.245654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:31.245778 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:31.246086 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:31.246092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:31.246553 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:31.246714 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.246723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.247782 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.247788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.248745 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.248758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.249470 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.249476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.249649 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.250179 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:31.250196 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:31.253387 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":251,"threads_started":1}
W20260812 06:52:31.253788 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:31.245235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:52431 (local address 127.17.69.126:44917)
0812 06:52:31.245438 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:31.245445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:31.245736 (+ 291us) server_negotiation.cc:407] Connection header received
0812 06:52:31.245888 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:31.245892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:31.245955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:31.246034 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:31.246851 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.247651 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.248883 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.249325 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.249433 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.253579 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:52431: BlockingRecv error: recv got EOF from 127.17.69.67:52431 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:31.570385 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:31.562191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:58609)
0812 06:52:31.562691 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:31.562707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:31.562820 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:31.563057 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:31.563060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:31.563423 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:31.563604 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.563614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.564575 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.564579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.565545 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.565557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.566190 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.566194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.566306 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.566819 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:31.566835 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:31.570122 (+ 3287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":268,"threads_started":1}
W20260812 06:52:31.570557 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:31.562286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:58609 (local address 127.17.69.126:44917)
0812 06:52:31.562524 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:31.562530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:31.562779 (+ 249us) server_negotiation.cc:407] Connection header received
0812 06:52:31.562901 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:31.562904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:31.562966 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:31.563043 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:31.563735 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.564462 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.565647 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.566083 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.566172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.570287 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:58609: BlockingRecv error: recv got EOF from 127.17.69.65:58609 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:31 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:31.949033 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:31.940782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:50201)
0812 06:52:31.941293 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:31.941308 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:31.941440 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:31.941757 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:31.941761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:31.942157 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:31.942325 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.942335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.943359 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.943363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.944332 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:31.944344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.945020 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.945026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:31.945188 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.945693 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:31.945709 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:31.948795 (+ 3086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":282,"threads_started":1}
W20260812 06:52:31.949172 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:31.940861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:50201 (local address 127.17.69.126:44917)
0812 06:52:31.941040 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:31.941046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:31.941388 (+ 342us) server_negotiation.cc:407] Connection header received
0812 06:52:31.941547 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:31.941552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:31.941622 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:31.941701 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:31.942501 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.943225 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.944468 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:31.944877 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:31.944983 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:31.948973 (+ 3990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:50201: BlockingRecv error: recv got EOF from 127.17.69.66:50201 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:32.267889 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:32.254559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:45623)
0812 06:52:32.255037 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:32.255052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:32.255162 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:32.255434 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:32.255438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:32.255792 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:32.255956 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.255966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.256974 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.256978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.257940 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.257952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.258664 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.258668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.258772 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.264278 (+ 5506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:32.264297 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:32.267565 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":258,"threads_started":1}
W20260812 06:52:32.268019 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:32.254705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45623 (local address 127.17.69.126:44917)
0812 06:52:32.254870 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:32.254874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:32.255152 (+ 278us) server_negotiation.cc:407] Connection header received
0812 06:52:32.255254 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:32.255258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:32.255323 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:32.255408 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:32.256114 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.256866 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.258071 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.258531 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.258643 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.267790 (+ 9147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:45623: BlockingRecv error: recv got EOF from 127.17.69.67:45623 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:32.579571 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:32.571239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:33117)
0812 06:52:32.571969 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:32.571989 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:32.572159 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:32.572496 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:32.572500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:32.572894 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:32.573066 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.573075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.573973 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.573977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.574890 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.574900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.575573 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.575577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.575696 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.576241 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:32.576259 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:32.579357 (+ 3098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":506,"threads_started":1}
W20260812 06:52:32.579717 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:32.571300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:33117 (local address 127.17.69.126:44917)
0812 06:52:32.571512 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:32.571519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:32.572070 (+ 551us) server_negotiation.cc:407] Connection header received
0812 06:52:32.572306 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:32.572311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:32.572384 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:32.572459 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:32.573186 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.573881 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.575015 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.575442 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.575532 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.579515 (+ 3983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:33117: BlockingRecv error: recv got EOF from 127.17.69.65:33117 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:32 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:32.958449 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:32.949895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:45999)
0812 06:52:32.950360 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:32.950381 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:32.950498 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:32.950866 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:32.950869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:32.951338 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:32.951540 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.951554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.952751 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.952755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.953651 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:32.953663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.954245 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.954265 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:32.954457 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.954993 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:32.955010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:32.958193 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":230,"threads_started":1}
W20260812 06:52:32.958599 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:32.949963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:45999 (local address 127.17.69.126:44917)
0812 06:52:32.950163 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:32.950169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:32.950454 (+ 285us) server_negotiation.cc:407] Connection header received
0812 06:52:32.950631 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:32.950637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:32.950720 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:32.950802 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:32.951693 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.952634 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.953757 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:32.954130 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:32.954214 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:32.958392 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:45999: BlockingRecv error: recv got EOF from 127.17.69.66:45999 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:33.277045 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:33.268819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:44731)
0812 06:52:33.269288 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:33.269305 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:33.269420 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:33.269670 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:33.269673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:33.270055 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:33.270248 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.270257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.271178 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.271182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.272079 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.272091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.272873 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.272879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.273049 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.273678 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:33.273701 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:33.276833 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":265,"threads_started":1}
W20260812 06:52:33.277191 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:33.268872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:44731 (local address 127.17.69.126:44917)
0812 06:52:33.269031 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:33.269035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:33.269369 (+ 334us) server_negotiation.cc:407] Connection header received
0812 06:52:33.269504 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:33.269508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:33.269566 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:33.269639 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:33.270371 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.271080 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.272196 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.272717 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.272834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.276976 (+ 4142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:44731: BlockingRecv error: recv got EOF from 127.17.69.67:44731 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:33.589073 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:33.580476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:60265)
0812 06:52:33.580974 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:33.580990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:33.581119 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:33.581515 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:33.581521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:33.581934 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:33.582108 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.582118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.583252 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.583256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.584161 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.584184 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.584933 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.584939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.585094 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.585664 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:33.585683 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:33.588833 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":252,"threads_started":1}
W20260812 06:52:33.589221 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:33.580493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:60265 (local address 127.17.69.126:44917)
0812 06:52:33.580698 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:33.580705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:33.581061 (+ 356us) server_negotiation.cc:407] Connection header received
0812 06:52:33.581299 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:33.581305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:33.581371 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:33.581454 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:33.582256 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.583118 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.584310 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.584763 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.584870 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.589002 (+ 4132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:60265: BlockingRecv error: recv got EOF from 127.17.69.65:60265 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:33 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:33.967978 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:33.959337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:47373)
0812 06:52:33.959885 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:33.959901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:33.960012 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:33.960285 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:33.960288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:33.960731 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:33.960906 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.960916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.961931 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.961937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.962950 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:33.962964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.963834 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.963840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:33.964034 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.964657 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:33.964676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:33.967660 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":287,"threads_started":1}
W20260812 06:52:33.968097 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:33.959385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:47373 (local address 127.17.69.126:44917)
0812 06:52:33.959544 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:33.959548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:33.959961 (+ 413us) server_negotiation.cc:407] Connection header received
0812 06:52:33.960093 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:33.960096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:33.960178 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:33.960255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:33.961012 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.961793 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.963090 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:33.963666 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:33.963788 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:33.967859 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:47373: BlockingRecv error: recv got EOF from 127.17.69.66:47373 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:34.286672 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:34.277825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:51781)
0812 06:52:34.278367 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:34.278383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:34.278511 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:34.278830 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:34.278835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:34.279250 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:34.279449 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.279459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.280513 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.280517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.281528 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.281545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.282397 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.282405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.282585 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.283108 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:34.283125 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:34.286421 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":283,"threads_started":1}
W20260812 06:52:34.286887 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:34.277912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:51781 (local address 127.17.69.126:44917)
0812 06:52:34.278112 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:34.278118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:34.278463 (+ 345us) server_negotiation.cc:407] Connection header received
0812 06:52:34.278634 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:34.278640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:34.278707 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:34.278786 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:34.279585 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.280375 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.281714 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.282191 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.282301 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.286618 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:51781: BlockingRecv error: recv got EOF from 127.17.69.67:51781 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:34.598085 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:34.589880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:43359)
0812 06:52:34.590332 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:34.590371 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:34.590492 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:34.590798 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:34.590800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:34.591175 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:34.591347 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.591357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.592293 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.592296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.593170 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.593182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.593822 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.593826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.593942 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.594522 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:34.594540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:34.597777 (+ 3237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":226,"threads_started":1}
W20260812 06:52:34.598296 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:34.589946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:43359 (local address 127.17.69.126:44917)
0812 06:52:34.590107 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:34.590111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:34.590434 (+ 323us) server_negotiation.cc:407] Connection header received
0812 06:52:34.590604 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:34.590607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:34.590683 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:34.590763 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:34.591461 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.592189 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.593300 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.593711 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.593799 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.598007 (+ 4208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:43359: BlockingRecv error: recv got EOF from 127.17.69.65:43359 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:34 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:34.977200 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:34.968876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:56969)
0812 06:52:34.969344 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:34.969359 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:34.969487 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:34.969825 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:34.969830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:34.970256 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:34.970443 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.970454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.971448 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.971454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.972364 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:34.972377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.973069 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.973075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:34.973252 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.973761 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:34.973778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:34.976896 (+ 3118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":240,"threads_started":1}
W20260812 06:52:34.977336 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:34.969047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:56969 (local address 127.17.69.126:44917)
0812 06:52:34.969277 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:34.969283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:34.969428 (+ 145us) server_negotiation.cc:407] Connection header received
0812 06:52:34.969618 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:34.969622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:34.969684 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:34.969769 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:34.970574 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.971306 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.972501 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:34.972932 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:34.973038 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:34.977112 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:56969: BlockingRecv error: recv got EOF from 127.17.69.66:56969 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.67@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:35.296243 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:35.287716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.67:40669)
0812 06:52:35.288140 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:35.288155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:35.288284 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:35.288584 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:35.288588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:35.289002 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:35.289151 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.289160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.290367 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.290373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.291602 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.291616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.292235 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.292239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.292358 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.292867 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:35.292884 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:35.296004 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
W20260812 06:52:35.296365 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:35.287754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:40669 (local address 127.17.69.126:44917)
0812 06:52:35.287932 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:35.287939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:35.288231 (+ 292us) server_negotiation.cc:407] Connection header received
0812 06:52:35.288401 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:35.288404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:35.288465 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:35.288541 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:35.289290 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.290173 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.291739 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.292122 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.292210 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.296167 (+ 3957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.67:40669: BlockingRecv error: recv got EOF from 127.17.69.67:40669 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.65@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:35.607856 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:35.599220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.65:52419)
0812 06:52:35.599691 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:35.599706 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:35.599832 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:35.600160 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:35.600165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:35.600586 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:35.600792 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.600806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.602033 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.602039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.602990 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.603002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.603769 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.603774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.603905 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.604463 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:35.604481 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:35.607611 (+ 3130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
W20260812 06:52:35.608035 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:35.599284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:52419 (local address 127.17.69.126:44917)
0812 06:52:35.599564 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:35.599570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:35.599793 (+ 223us) server_negotiation.cc:407] Connection header received
0812 06:52:35.599943 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:35.599949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:35.600033 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:35.600112 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:35.600954 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.601892 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.603134 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.603649 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.603744 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.607805 (+ 4061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.65:52419: BlockingRecv error: recv got EOF from 127.17.69.65:52419 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 06:52:35 dist-test-slave-6zbq krb5kdc[32614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.69.66@KRBTEST.COM for oryx/127.17.69.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 06:52:35.986933 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:35.978409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:44917 (local address 127.17.69.66:50557)
0812 06:52:35.978893 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:35.978910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:35.979023 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:35.979309 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:35.979312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:35.979675 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:35.979872 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.979882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.980941 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.980946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.982231 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:35.982247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.982907 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.982912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:35.983041 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.983582 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:35.983599 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:35.986686 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.69.126:44917: Server oryx/127.17.69.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
W20260812 06:52:35.987077 1961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:35.978496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:50557 (local address 127.17.69.126:44917)
0812 06:52:35.978674 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:35.978678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:35.978973 (+ 295us) server_negotiation.cc:407] Connection header received
0812 06:52:35.979135 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:35.979139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:35.979197 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:35.979283 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:35.979979 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.980783 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.982379 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:35.982795 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:35.982884 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:35.986868 (+ 3984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.69.66:50557: BlockingRecv error: recv got EOF from 127.17.69.66:50557 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:52:36.131186 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1178
I20260812 06:52:36.142871 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:36.132164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.17.69.65:34055)
0812 06:52:36.132722 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:36.132729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:36.132755 (+ 26us) server_negotiation.cc:407] Connection header received
0812 06:52:36.132821 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:36.132828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:36.133056 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:36.133207 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:36.134583 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.135586 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.136710 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.137362 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.137468 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:36.139767 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:36.139775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:36.139790 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:36.139835 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:36.142327 (+ 2492us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:52:36.142650 (+ 323us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60520: Request ticket server kudu/127.17.69.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":292,"threads_started":1}
W20260812 06:52:36.142972 2041 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60520: Request ticket server kudu/127.17.69.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:52:36.143024 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1178: Not authorized: Client connection negotiation failed: client connection to 127.17.69.65:34055: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:52:36.143217 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 1178
I20260812 06:52:36.169907 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1478
I20260812 06:52:36.181685 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:36.170853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.17.69.66:42431)
0812 06:52:36.171478 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:36.171485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:36.171515 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:52:36.171583 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:36.171589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:36.171821 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:36.171978 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:36.173528 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.174657 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.175781 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.176313 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.176407 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:36.178626 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:36.178635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:36.178647 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:36.178689 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:36.181142 (+ 2453us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:52:36.181421 (+ 279us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45798: Request ticket server kudu/127.17.69.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":311,"threads_started":1}
W20260812 06:52:36.181681 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1478: Not authorized: Client connection negotiation failed: client connection to 127.17.69.66:42431: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:52:36.181770 2042 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45798: Request ticket server kudu/127.17.69.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:52:36.181831 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 1478
I20260812 06:52:36.200814 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1718
I20260812 06:52:36.212139 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:36.201783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.17.69.67:43517)
0812 06:52:36.202275 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:36.202283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:36.202308 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:52:36.202408 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:36.202415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:36.202634 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:36.202824 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:36.204084 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.205072 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.206238 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.206833 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.206951 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:36.209259 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:36.209267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:36.209281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:36.209331 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:36.211528 (+ 2197us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 06:52:36.211846 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50032: Request ticket server kudu/127.17.69.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":219,"threads_started":1}
W20260812 06:52:36.212144 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1718: Not authorized: Client connection negotiation failed: client connection to 127.17.69.67:43517: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.69.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 06:52:36.212225 2043 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50032: Request ticket server kudu/127.17.69.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 06:52:36.212257 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 1718
I20260812 06:52:36.234020 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 1074
I20260812 06:52:36.235062 1170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:36.348910 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 1074
2026-08-12T06:52:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21054 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T06:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:36Z Disabled control of system clock
I20260812 06:52:36.432600 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40407
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 06:52:36.745865 2051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:36.746100 2051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:36.746212 2051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:36.753129 2051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:36.753216 2051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:36.753278 2051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:36.753351 2051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:36.753412 2051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:36.762318 2051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:36.763418 2051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:36.764991 2051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:36.775583 2059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:36.775662 2064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:36.776193 2060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:36.776568 2051 server_base.cc:1034] running on GCE node
I20260812 06:52:36.777213 2051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:36.778456 2051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:36.779647 2051 hybrid_clock.cc:648] HybridClock initialized: now 1786517556779606 us; error 37 us; skew 500 ppm
I20260812 06:52:36.780009 2051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:36.781981 2051 webserver.cc:460] Webserver started at http://127.17.69.126:35877/ using document root <none> and password file <none>
I20260812 06:52:36.782629 2051 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:36.782728 2051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:36.783023 2051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:36.785605 2051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f"
format_stamp: "Formatted at 2026-08-12 06:52:36 on dist-test-slave-6zbq"
server_key: "c0eb0454bd60bfcae58f259d241196be"
server_key_iv: "fe21ba85e33cbd1eddecec2dced29886"
server_key_version: "encryptionkey@0"
I20260812 06:52:36.786325 2051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f"
format_stamp: "Formatted at 2026-08-12 06:52:36 on dist-test-slave-6zbq"
server_key: "c0eb0454bd60bfcae58f259d241196be"
server_key_iv: "fe21ba85e33cbd1eddecec2dced29886"
server_key_version: "encryptionkey@0"
I20260812 06:52:36.790937 2051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:52:36.794286 2073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:36.795487 2051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:36.795642 2051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f"
format_stamp: "Formatted at 2026-08-12 06:52:36 on dist-test-slave-6zbq"
server_key: "c0eb0454bd60bfcae58f259d241196be"
server_key_iv: "fe21ba85e33cbd1eddecec2dced29886"
server_key_version: "encryptionkey@0"
I20260812 06:52:36.795835 2051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:36.815577 2051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:36.816265 2051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:36.816504 2051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:36.832978 2051 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40407
I20260812 06:52:36.833010 2159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40407 every 8 connection(s)
I20260812 06:52:36.834102 2051 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:36.837425 2160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:36.840896 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 2051
I20260812 06:52:36.841095 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:52:36.841676 17685 external_mini_cluster.cc:1428] Setting key eac12e7e974a95e0cfa50fb70e3bbc94
I20260812 06:52:36.847606 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f: Bootstrap starting.
I20260812 06:52:36.850901 2160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:36.852038 2160 log.cc:824] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:36.854856 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:36.843439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.17.69.126:40407)
0812 06:52:36.844293 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:36.844302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:36.844343 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:52:36.846222 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:36.846242 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:36.846666 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:36.846949 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:36.847592 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.848826 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.850640 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:36.851311 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:36.851432 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:36.852625 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:36.852636 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:36.852653 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:36.853334 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:36.853414 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:36.853435 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:36.853513 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:36.853730 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 06:52:36.854002 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":324,"threads_started":1}
I20260812 06:52:36.854965 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f: No bootstrap required, opened a new log
I20260812 06:52:36.861771 2160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } }
I20260812 06:52:36.862089 2160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:36.862190 2160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc0f3499ea40b2b96cc62a11bd9d7f, State: Initialized, Role: FOLLOWER
I20260812 06:52:36.862710 2160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } }
I20260812 06:52:36.862918 2160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:36.863037 2160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:36.863199 2160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:36.865177 2160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } }
I20260812 06:52:36.865618 2160 leader_election.cc:302] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10fc0f3499ea40b2b96cc62a11bd9d7f; no voters:
I20260812 06:52:36.866199 2160 leader_election.cc:288] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:52:36.866302 2166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:36.866958 2166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [term 1 LEADER]: Becoming Leader. State: Replica: 10fc0f3499ea40b2b96cc62a11bd9d7f, State: Running, Role: LEADER
I20260812 06:52:36.867431 2166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } }
I20260812 06:52:36.868039 2160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:36.869966 2169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fc0f3499ea40b2b96cc62a11bd9d7f. Latest consensus state: current_term: 1 leader_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } } }
I20260812 06:52:36.869989 2168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fc0f3499ea40b2b96cc62a11bd9d7f" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40407 } } }
I20260812 06:52:36.870507 2168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:36.870502 2169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:36.872156 2177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:36.876873 2177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:36.884981 2177 catalog_manager.cc:1347] Generated new cluster ID: f7e06ea952314cd8bb8ca363cfbfe69a
I20260812 06:52:36.885071 2177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:36.901984 2177 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:52:36.903635 2177 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:36.910646 2177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10fc0f3499ea40b2b96cc62a11bd9d7f: Generated new TSK 0
I20260812 06:52:36.911505 2177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:52:36.926813 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--builtin_ntp_servers=127.17.69.84:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 06:52:37.246873 2199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:37.247125 2199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:37.247227 2199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:37.254235 2199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:37.254326 2199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:37.254513 2199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:37.263172 2199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:37.264331 2199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:37.266043 2199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:37.278285 2208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:37.278232 2214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:37.278200 2211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:37.279524 2199 server_base.cc:1034] running on GCE node
I20260812 06:52:37.280058 2199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:37.280876 2199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:37.282070 2199 hybrid_clock.cc:648] HybridClock initialized: now 1786517557282009 us; error 55 us; skew 500 ppm
I20260812 06:52:37.282375 2199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:37.288600 2199 webserver.cc:460] Webserver started at http://127.17.69.65:45493/ using document root <none> and password file <none>
I20260812 06:52:37.289219 2199 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:37.289330 2199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:37.289631 2199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:37.292378 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "f7406a25262340b4a37eaa2fa21a7579"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "9bf4fd0e929229a51779e87a7d326644"
server_key_iv: "1d696b4eda02b5ac45a03afe4fc63319"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.293107 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "f7406a25262340b4a37eaa2fa21a7579"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "9bf4fd0e929229a51779e87a7d326644"
server_key_iv: "1d696b4eda02b5ac45a03afe4fc63319"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.297756 2199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 06:52:37.301023 2226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.302237 2199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:37.302405 2199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "f7406a25262340b4a37eaa2fa21a7579"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "9bf4fd0e929229a51779e87a7d326644"
server_key_iv: "1d696b4eda02b5ac45a03afe4fc63319"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.302593 2199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:37.326330 2199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:37.327050 2199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:37.327318 2199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:37.328298 2199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:37.329638 2199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:37.329735 2199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.329840 2199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:37.329911 2199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.360764 2199 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:40703
I20260812 06:52:37.360837 2424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:40703 every 8 connection(s)
I20260812 06:52:37.361922 2199 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:37.363185 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 2199
I20260812 06:52:37.363392 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:52:37.364183 17685 external_mini_cluster.cc:1428] Setting key b1ded724b8b8038f3d53c25057184c6e
I20260812 06:52:37.366885 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--builtin_ntp_servers=127.17.69.84:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:52:37.374634 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:37.364645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:51809 (local address 127.17.69.126:40407)
0812 06:52:37.365088 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:37.365093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:37.366392 (+ 1299us) server_negotiation.cc:407] Connection header received
0812 06:52:37.368153 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:37.368158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:37.368252 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:37.368355 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:37.369458 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.370256 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.372015 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.372415 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.372532 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:37.373798 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:37.373803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:37.373807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:37.373824 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:37.373864 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:37.373876 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:37.373949 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:37.374171 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 06:52:37.374268 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":227,"threads_started":1}
I20260812 06:52:37.375495 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:37.365205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40407 (local address 127.17.69.65:51809)
0812 06:52:37.366210 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:37.366253 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:37.367904 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:37.368631 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:37.368642 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:37.368744 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:37.369291 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:37.369308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.370437 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.370441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:37.371869 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:37.371882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.372637 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.372645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:37.372903 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:37.373508 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:37.373511 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:37.373544 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:37.373620 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:37.373962 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:37.373974 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:37.373982 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:52:37.374149 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 06:52:37.374540 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":300,"threads_started":1}
I20260812 06:52:37.376946 2425 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40407
I20260812 06:52:37.377240 2425 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:37.378046 2425 heartbeater.cc:511] Master 127.17.69.126:40407 requested a full tablet report, sending...
I20260812 06:52:37.380122 2101 ts_manager.cc:194] Registered new tserver with Master: f7406a25262340b4a37eaa2fa21a7579 (127.17.69.65:40703)
I20260812 06:52:37.382005 2101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.65:51809
W20260812 06:52:37.686295 2430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:37.686573 2430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:37.686661 2430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:37.693482 2430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:37.693562 2430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:37.693656 2430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:37.702211 2430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:37.703279 2430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:37.704901 2430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:37.716918 2440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:37.716872 2441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:37.716887 2447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:37.717474 2430 server_base.cc:1034] running on GCE node
I20260812 06:52:37.718055 2430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:37.718953 2430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:37.720148 2430 hybrid_clock.cc:648] HybridClock initialized: now 1786517557720090 us; error 47 us; skew 500 ppm
I20260812 06:52:37.720440 2430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:37.722525 2430 webserver.cc:460] Webserver started at http://127.17.69.66:40991/ using document root <none> and password file <none>
I20260812 06:52:37.723171 2430 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:37.723270 2430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:37.723563 2430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:37.726135 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "c4ef66298c7a4396891c09bfb28f1062"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "d3a1dfc0533dfc5c60f68e10282b240b"
server_key_iv: "0acd8bdbafe23aecbb96d2ff40695d86"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.726912 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "c4ef66298c7a4396891c09bfb28f1062"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "d3a1dfc0533dfc5c60f68e10282b240b"
server_key_iv: "0acd8bdbafe23aecbb96d2ff40695d86"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.731513 2430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 06:52:37.734730 2457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.736130 2430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:37.736263 2430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "c4ef66298c7a4396891c09bfb28f1062"
format_stamp: "Formatted at 2026-08-12 06:52:37 on dist-test-slave-6zbq"
server_key: "d3a1dfc0533dfc5c60f68e10282b240b"
server_key_iv: "0acd8bdbafe23aecbb96d2ff40695d86"
server_key_version: "encryptionkey@0"
I20260812 06:52:37.736430 2430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:37.773170 2430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:37.773883 2430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:37.774118 2430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:37.774996 2430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:37.776293 2430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:37.776377 2430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.776451 2430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:37.776504 2430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:37.805680 2430 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:33059
I20260812 06:52:37.805752 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:33059 every 8 connection(s)
I20260812 06:52:37.806931 2430 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:37.814286 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 2430
I20260812 06:52:37.814507 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:52:37.814927 17685 external_mini_cluster.cc:1428] Setting key f98bf5ea7917d6764adca43a02010e21
I20260812 06:52:37.817571 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--builtin_ntp_servers=127.17.69.84:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 06:52:37.818635 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:37.809177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:56477 (local address 127.17.69.126:40407)
0812 06:52:37.809390 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:37.809397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:37.810710 (+ 1313us) server_negotiation.cc:407] Connection header received
0812 06:52:37.812347 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:37.812351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:37.812420 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:37.812493 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:37.813395 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.814217 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.815901 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.816342 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.816466 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:37.817804 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:37.817811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:37.817815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:37.817836 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:37.817874 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:37.817888 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:37.817972 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:37.818212 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 06:52:37.818307 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:52:37.819453 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:37.809695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40407 (local address 127.17.69.66:56477)
0812 06:52:37.810572 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:37.810613 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:37.812116 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:37.812656 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:37.812663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:37.812744 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:37.813271 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:37.813287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.814371 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.814376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:37.815756 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:37.815769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:37.816562 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:37.816570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:37.816848 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:37.817504 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:37.817507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:37.817542 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:37.817614 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:37.818009 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:37.818018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:37.818025 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:52:37.818181 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 06:52:37.818485 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":258,"threads_started":1}
I20260812 06:52:37.820679 2659 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40407
I20260812 06:52:37.820933 2659 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:37.821630 2659 heartbeater.cc:511] Master 127.17.69.126:40407 requested a full tablet report, sending...
I20260812 06:52:37.822842 2101 ts_manager.cc:194] Registered new tserver with Master: c4ef66298c7a4396891c09bfb28f1062 (127.17.69.66:33059)
I20260812 06:52:37.823618 2101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.66:56477
W20260812 06:52:38.128929 2664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:38.129175 2664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:38.129272 2664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:38.136245 2664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:38.136327 2664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:38.136454 2664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:38.145004 2664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:38.146080 2664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:38.147709 2664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:38.159684 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:38.159960 2674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:38.160147 2679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:38.160665 2664 server_base.cc:1034] running on GCE node
I20260812 06:52:38.161214 2664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:38.162055 2664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:38.163242 2664 hybrid_clock.cc:648] HybridClock initialized: now 1786517558163185 us; error 44 us; skew 500 ppm
I20260812 06:52:38.163522 2664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:38.165494 2664 webserver.cc:460] Webserver started at http://127.17.69.67:35553/ using document root <none> and password file <none>
I20260812 06:52:38.166105 2664 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:38.166194 2664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:38.166512 2664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:38.169175 2664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "0ce748fa2b2341c7ada7da16eb88321b"
format_stamp: "Formatted at 2026-08-12 06:52:38 on dist-test-slave-6zbq"
server_key: "2f2314a0635eb1cb0f75ed0c0073727c"
server_key_iv: "fa00b2288cb150449c8ffe7aa2f7db87"
server_key_version: "encryptionkey@0"
I20260812 06:52:38.169947 2664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "0ce748fa2b2341c7ada7da16eb88321b"
format_stamp: "Formatted at 2026-08-12 06:52:38 on dist-test-slave-6zbq"
server_key: "2f2314a0635eb1cb0f75ed0c0073727c"
server_key_iv: "fa00b2288cb150449c8ffe7aa2f7db87"
server_key_version: "encryptionkey@0"
I20260812 06:52:38.174904 2664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260812 06:52:38.177996 2688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:38.179215 2664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 06:52:38.179361 2664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "0ce748fa2b2341c7ada7da16eb88321b"
format_stamp: "Formatted at 2026-08-12 06:52:38 on dist-test-slave-6zbq"
server_key: "2f2314a0635eb1cb0f75ed0c0073727c"
server_key_iv: "fa00b2288cb150449c8ffe7aa2f7db87"
server_key_version: "encryptionkey@0"
I20260812 06:52:38.179543 2664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:38.203970 2664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:38.204672 2664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:38.204936 2664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:38.205849 2664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:38.207194 2664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:38.207285 2664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:38.207386 2664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:38.207456 2664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:38.236105 2664 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41785
I20260812 06:52:38.236186 2912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41785 every 8 connection(s)
I20260812 06:52:38.237244 2664 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:38.242647 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 2664
I20260812 06:52:38.242816 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:38.243202 17685 external_mini_cluster.cc:1428] Setting key 05093e8a49749be1255fc7262a595856
I20260812 06:52:38.248855 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:38.239532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:53067 (local address 127.17.69.126:40407)
0812 06:52:38.239707 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:38.239712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:38.241023 (+ 1311us) server_negotiation.cc:407] Connection header received
0812 06:52:38.242805 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:38.242810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:38.242895 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:38.242988 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:38.244094 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.245034 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.246536 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.246985 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.247082 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:38.248248 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:38.248253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:38.248256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:38.248273 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:38.248306 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:38.248319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:38.248392 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:38.248551 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 06:52:38.248628 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:52:38.249738 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:38.240048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40407 (local address 127.17.69.67:53067)
0812 06:52:38.240881 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:38.240914 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:38.242555 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:38.243205 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:38.243213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:38.243305 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:38.243934 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:38.243954 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.245202 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.245206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:38.246390 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:38.246406 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.247198 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.247204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:38.247388 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:38.247937 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:38.247940 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:38.247974 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:38.248045 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:38.248402 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:38.248408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:38.248413 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:52:38.248527 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 06:52:38.248782 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":270,"threads_started":1}
I20260812 06:52:38.250908 2913 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40407
I20260812 06:52:38.251165 2913 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:38.251806 2913 heartbeater.cc:511] Master 127.17.69.126:40407 requested a full tablet report, sending...
I20260812 06:52:38.253005 2101 ts_manager.cc:194] Registered new tserver with Master: 0ce748fa2b2341c7ada7da16eb88321b (127.17.69.67:41785)
I20260812 06:52:38.253746 2101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.67:53067
I20260812 06:52:38.257956 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:52:38.263474 2427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:38.262107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.17.69.126:40407)
0812 06:52:38.262265 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:38.262269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:38.262775 (+ 506us) server_negotiation.cc:407] Connection header received
0812 06:52:38.262904 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:38.262908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:38.262974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:38.263043 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:38.263318 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40510: BlockingRecv error: recv got EOF from 127.0.0.1:40510 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:52:38.264256 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 2199
I20260812 06:52:38.271423 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:38.265081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.17.69.65:40703)
0812 06:52:38.265575 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:38.265582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:38.265603 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:52:38.265656 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:38.265661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:38.265814 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:38.265941 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:38.266363 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.267308 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.268515 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.269069 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.269208 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:38.270079 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:38.270086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:38.270096 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:38.270639 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:38.270682 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:38.270701 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:38.270797 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:38.270963 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 06:52:38.271086 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260812 06:52:38.272462 2416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:38.354949 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 2199
I20260812 06:52:38.373185 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 2430
I20260812 06:52:38.380370 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:38.374051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.17.69.66:33059)
0812 06:52:38.374601 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:38.374608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:38.374635 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:52:38.374695 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:38.374700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:38.374918 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:38.375041 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:38.375375 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.376371 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.377528 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.378074 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.378168 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:38.379050 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:38.379058 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:38.379070 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:38.379641 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:38.379688 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:38.379708 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:38.379793 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:38.379900 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 06:52:38.380026 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":295,"threads_started":1}
I20260812 06:52:38.381531 2650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:38.461138 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 2430
I20260812 06:52:38.484346 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 2664
I20260812 06:52:38.492162 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:38.485241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.17.69.67:41785)
0812 06:52:38.485840 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:38.485846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:38.485877 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:52:38.485940 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:38.485947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:38.486153 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:38.486306 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:38.486821 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.487828 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.489082 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:38.489628 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:38.489751 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:38.490780 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:38.490789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:38.490801 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:38.491367 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:38.491414 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:38.491434 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:38.491533 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:38.491700 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:52:38.491827 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":300,"threads_started":1}
I20260812 06:52:38.493350 2905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:38.572849 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 2664
I20260812 06:52:38.590116 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 2051
I20260812 06:52:38.591311 2153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:38.692418 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 2051
2026-08-12T06:52:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2337 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T06:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:38Z Disabled control of system clock
I20260812 06:52:38.772136 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:39755
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:39755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:52:39.082472 2954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:39.082744 2954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:39.082873 2954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:39.089637 2954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:39.089740 2954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:39.089807 2954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:39.089884 2954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:39.089944 2954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:39.098613 2954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:39755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:39755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:39.099721 2954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:39.101372 2954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:39.111783 2965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:39.111876 2966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:39.112605 2968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:39.112905 2954 server_base.cc:1034] running on GCE node
I20260812 06:52:39.113564 2954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:39.114835 2954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:39.116027 2954 hybrid_clock.cc:648] HybridClock initialized: now 1786517559115988 us; error 32 us; skew 500 ppm
I20260812 06:52:39.116343 2954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:39.118333 2954 webserver.cc:460] Webserver started at http://127.17.69.126:40523/ using document root <none> and password file <none>
I20260812 06:52:39.118984 2954 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:39.119081 2954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:39.119380 2954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:39.122237 2954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "559783e3037e4a5fa29e6e453aabce1b"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "8d854807c30e2323e611720184c8d921"
server_key_iv: "099f6e345b0833a7cd4ba4580eb2c2fb"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.123020 2954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "559783e3037e4a5fa29e6e453aabce1b"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "8d854807c30e2323e611720184c8d921"
server_key_iv: "099f6e345b0833a7cd4ba4580eb2c2fb"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.127615 2954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:52:39.130744 2976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:39.131880 2954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 06:52:39.132035 2954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "559783e3037e4a5fa29e6e453aabce1b"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "8d854807c30e2323e611720184c8d921"
server_key_iv: "099f6e345b0833a7cd4ba4580eb2c2fb"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.132232 2954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:39.137603 2954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:39.138281 2954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:39.138561 2954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:39.154920 2954 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:39755
I20260812 06:52:39.154958 3051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:39755 every 8 connection(s)
I20260812 06:52:39.156036 2954 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:39.159349 3052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:39.160110 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 2954
I20260812 06:52:39.160306 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:52:39.161152 17685 external_mini_cluster.cc:1428] Setting key a7af622de9240909cc3b582baee2f30b
I20260812 06:52:39.168869 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:39.163094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.17.69.126:39755)
0812 06:52:39.164038 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:39.164048 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:39.164086 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:52:39.165616 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:39.165639 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:39.166013 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:39.166394 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:39.167390 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:39.167396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:39.167409 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:39.167431 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:39.167486 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:39.167501 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:39.167715 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:39.167845 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 06:52:39.168050 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":316,"threads_started":1}
I20260812 06:52:39.171689 3052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b: Bootstrap starting.
I20260812 06:52:39.174314 3052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:39.175271 3052 log.cc:824] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:39.177706 3052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b: No bootstrap required, opened a new log
I20260812 06:52:39.184556 3052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } }
I20260812 06:52:39.184844 3052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:39.184935 3052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559783e3037e4a5fa29e6e453aabce1b, State: Initialized, Role: FOLLOWER
I20260812 06:52:39.185448 3052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } }
I20260812 06:52:39.185653 3052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:39.185776 3052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:39.185925 3052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:39.187887 3052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } }
I20260812 06:52:39.188324 3052 leader_election.cc:302] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 559783e3037e4a5fa29e6e453aabce1b; no voters:
I20260812 06:52:39.188817 3052 leader_election.cc:288] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:52:39.188941 3058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:39.189523 3058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [term 1 LEADER]: Becoming Leader. State: Replica: 559783e3037e4a5fa29e6e453aabce1b, State: Running, Role: LEADER
I20260812 06:52:39.190042 3058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } }
I20260812 06:52:39.190600 3052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:39.193204 3060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 559783e3037e4a5fa29e6e453aabce1b. Latest consensus state: current_term: 1 leader_uuid: "559783e3037e4a5fa29e6e453aabce1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } } }
I20260812 06:52:39.193269 3059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "559783e3037e4a5fa29e6e453aabce1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559783e3037e4a5fa29e6e453aabce1b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 39755 } } }
I20260812 06:52:39.193648 3060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:39.193794 3059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:39.205538 3071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:39.210457 3071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:39.217902 3071 catalog_manager.cc:1347] Generated new cluster ID: 27c268950d3e4b1881c9bbc3960f2be3
I20260812 06:52:39.218003 3071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:39.229028 3071 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:52:39.229393 3071 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:39.234279 3071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 559783e3037e4a5fa29e6e453aabce1b: Generated new TSK 0
I20260812 06:52:39.235049 3071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:52:39.239691 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--builtin_ntp_servers=127.17.69.84:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 06:52:39.561203 3092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:39.561470 3092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:39.561596 3092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:39.568568 3092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:39.568667 3092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:39.568813 3092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:39.577445 3092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:39.578563 3092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:39.580183 3092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:39.591974 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:39.592094 3101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:39.592083 3102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:39.593150 3092 server_base.cc:1034] running on GCE node
I20260812 06:52:39.593708 3092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:39.594568 3092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:39.595759 3092 hybrid_clock.cc:648] HybridClock initialized: now 1786517559595720 us; error 34 us; skew 500 ppm
I20260812 06:52:39.596040 3092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:39.598037 3092 webserver.cc:460] Webserver started at http://127.17.69.65:42713/ using document root <none> and password file <none>
I20260812 06:52:39.598716 3092 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:39.598814 3092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:39.599100 3092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:39.603653 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "896f716c1ebf4a51adc726901dcb5415"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "e46b9c3559db09b277ca490cef85bb38"
server_key_iv: "13b67f775d470a4eee912d8af0173a88"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.604399 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "896f716c1ebf4a51adc726901dcb5415"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "e46b9c3559db09b277ca490cef85bb38"
server_key_iv: "13b67f775d470a4eee912d8af0173a88"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.609112 3092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:52:39.612200 3118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:39.613338 3092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:52:39.613487 3092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "896f716c1ebf4a51adc726901dcb5415"
format_stamp: "Formatted at 2026-08-12 06:52:39 on dist-test-slave-6zbq"
server_key: "e46b9c3559db09b277ca490cef85bb38"
server_key_iv: "13b67f775d470a4eee912d8af0173a88"
server_key_version: "encryptionkey@0"
I20260812 06:52:39.613675 3092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:39.618906 3092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:39.619586 3092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:39.619817 3092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:39.620621 3092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:39.621910 3092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:39.622004 3092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:39.622099 3092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:39.622165 3092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:39.651540 3092 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:37017
I20260812 06:52:39.651602 3320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:37017 every 8 connection(s)
I20260812 06:52:39.652683 3092 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:39.656279 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 3092
I20260812 06:52:39.656481 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:52:39.657305 17685 external_mini_cluster.cc:1428] Setting key ce41b61f73f123985de06326c5af9112
I20260812 06:52:39.660367 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--builtin_ntp_servers=127.17.69.84:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 06:52:39.660993 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:39.654921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:59427 (local address 127.17.69.126:39755)
0812 06:52:39.655124 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:39.655130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:39.656621 (+ 1491us) server_negotiation.cc:407] Connection header received
0812 06:52:39.658711 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:39.658717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:39.658777 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:39.658883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:39.660360 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:39.660364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:39.660368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:39.660384 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:39.660416 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:39.660427 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:39.660540 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:39.660745 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 06:52:39.660798 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:52:39.662185 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:39.655426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39755 (local address 127.17.69.65:59427)
0812 06:52:39.656453 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:39.656509 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:39.658464 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:39.659183 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:39.659191 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:39.659296 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:39.659990 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:39.659994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:39.660045 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:39.660176 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:39.660532 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:39.660544 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:39.660552 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:52:39.660811 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 06:52:39.661054 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260812 06:52:39.663496 3324 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39755
I20260812 06:52:39.663791 3324 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:39.664426 3324 heartbeater.cc:511] Master 127.17.69.126:39755 requested a full tablet report, sending...
I20260812 06:52:39.666404 3002 ts_manager.cc:194] Registered new tserver with Master: 896f716c1ebf4a51adc726901dcb5415 (127.17.69.65:37017)
W20260812 06:52:39.981865 3332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:39.982111 3332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:39.982223 3332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:39.989254 3332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:39.989348 3332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:39.989490 3332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:39.998335 3332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:39.999526 3332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:40.001260 3332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:40.013764 3345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:40.013803 3349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:40.013851 3346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:40.014643 3332 server_base.cc:1034] running on GCE node
I20260812 06:52:40.015245 3332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:40.016134 3332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:40.017309 3332 hybrid_clock.cc:648] HybridClock initialized: now 1786517560017257 us; error 45 us; skew 500 ppm
I20260812 06:52:40.017598 3332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:40.019726 3332 webserver.cc:460] Webserver started at http://127.17.69.66:45327/ using document root <none> and password file <none>
I20260812 06:52:40.020354 3332 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:40.020440 3332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:40.020735 3332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:40.023609 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "5726b8d40bc041db96696e292641b2be"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "577d1025c1f3de52ca61cf6bb4e6a209"
server_key_iv: "0621512c02f13a2d9384ab6792b65316"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.024376 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "5726b8d40bc041db96696e292641b2be"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "577d1025c1f3de52ca61cf6bb4e6a209"
server_key_iv: "0621512c02f13a2d9384ab6792b65316"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.029397 3332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:52:40.032615 3358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.033804 3332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:40.033950 3332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "5726b8d40bc041db96696e292641b2be"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "577d1025c1f3de52ca61cf6bb4e6a209"
server_key_iv: "0621512c02f13a2d9384ab6792b65316"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.034138 3332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:40.039471 3332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:40.040191 3332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:40.040426 3332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:40.041307 3332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:40.042640 3332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:40.042735 3332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.042841 3332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:40.042910 3332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.071419 3332 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:33439
I20260812 06:52:40.071476 3571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:33439 every 8 connection(s)
I20260812 06:52:40.072693 3332 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:40.076767 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 3332
I20260812 06:52:40.076908 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:52:40.077350 17685 external_mini_cluster.cc:1428] Setting key 7d573a0febd9f478e04be5419ecc8823
I20260812 06:52:40.079927 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--builtin_ntp_servers=127.17.69.84:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 06:52:40.080214 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.075022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:49359 (local address 127.17.69.126:39755)
0812 06:52:40.075186 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.075190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.076475 (+ 1285us) server_negotiation.cc:407] Connection header received
0812 06:52:40.078151 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.078158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.078222 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.078314 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.079529 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:40.079535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:40.079540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:40.079557 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:40.079588 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:40.079599 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:40.079697 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:40.079873 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 06:52:40.079938 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:52:40.080996 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.075470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39755 (local address 127.17.69.66:49359)
0812 06:52:40.076326 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:40.076366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:40.077902 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:40.078561 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:40.078569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:40.078650 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:40.079250 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:40.079253 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:40.079297 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:40.079376 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:40.079681 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:40.079691 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:40.079699 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:52:40.079938 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 06:52:40.080163 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":262,"threads_started":1}
I20260812 06:52:40.082136 3573 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39755
I20260812 06:52:40.082461 3573 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:40.083078 3573 heartbeater.cc:511] Master 127.17.69.126:39755 requested a full tablet report, sending...
I20260812 06:52:40.084455 3002 ts_manager.cc:194] Registered new tserver with Master: 5726b8d40bc041db96696e292641b2be (127.17.69.66:33439)
W20260812 06:52:40.408831 3576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:40.409158 3576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:40.409265 3576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:40.416178 3576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:40.416261 3576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:40.416363 3576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:40.425051 3576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:39755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:40.426179 3576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:40.427845 3576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:40.440165 3592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:40.440219 3586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:40.440244 3585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:40.443512 3576 server_base.cc:1034] running on GCE node
I20260812 06:52:40.444046 3576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:40.444864 3576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:40.446043 3576 hybrid_clock.cc:648] HybridClock initialized: now 1786517560445992 us; error 45 us; skew 500 ppm
I20260812 06:52:40.446313 3576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:52:40.448345 3576 webserver.cc:460] Webserver started at http://127.17.69.67:44451/ using document root <none> and password file <none>
I20260812 06:52:40.448968 3576 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:40.449060 3576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:40.449357 3576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:40.452014 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "d112e9fd3d5f4a12b65bc089943095a9"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "507b92041aae6a1a9cd92b8115cddee9"
server_key_iv: "e2c6cef7b4ef6537c6a1f2db4b2406e6"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.452781 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "d112e9fd3d5f4a12b65bc089943095a9"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "507b92041aae6a1a9cd92b8115cddee9"
server_key_iv: "e2c6cef7b4ef6537c6a1f2db4b2406e6"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.457479 3576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:52:40.460563 3604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.461704 3576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:52:40.461855 3576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "d112e9fd3d5f4a12b65bc089943095a9"
format_stamp: "Formatted at 2026-08-12 06:52:40 on dist-test-slave-6zbq"
server_key: "507b92041aae6a1a9cd92b8115cddee9"
server_key_iv: "e2c6cef7b4ef6537c6a1f2db4b2406e6"
server_key_version: "encryptionkey@0"
I20260812 06:52:40.462051 3576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:40.467257 3576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:40.467952 3576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:40.468189 3576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:40.469019 3576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:40.470299 3576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:40.470448 3576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.470546 3576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:40.470618 3576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:40.499665 3576 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41233
I20260812 06:52:40.499708 3820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41233 every 8 connection(s)
I20260812 06:52:40.500808 3576 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:40.506735 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 3576
I20260812 06:52:40.506894 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:40.507326 17685 external_mini_cluster.cc:1428] Setting key 7a51b82e30844030b6f301ab3fe7f4c3
I20260812 06:52:40.508267 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.503093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:56971 (local address 127.17.69.126:39755)
0812 06:52:40.503263 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.503268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.504502 (+ 1234us) server_negotiation.cc:407] Connection header received
0812 06:52:40.506149 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.506154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.506216 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.506310 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.507560 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:40.507566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:40.507571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:40.507591 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:40.507621 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:40.507633 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:40.507746 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:40.507922 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 06:52:40.507991 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:52:40.509066 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.503527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:39755 (local address 127.17.69.67:56971)
0812 06:52:40.504353 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:40.504391 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:40.505911 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:40.506539 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:40.506546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:40.506628 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:40.507284 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:52:40.507287 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:40.507321 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:52:40.507398 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:40.507716 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:40.507727 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:40.507734 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:52:40.507988 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 06:52:40.508211 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":255,"threads_started":1}
I20260812 06:52:40.510387 3821 heartbeater.cc:347] Connected to a master server at 127.17.69.126:39755
I20260812 06:52:40.510640 3821 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:40.511260 3821 heartbeater.cc:511] Master 127.17.69.126:39755 requested a full tablet report, sending...
I20260812 06:52:40.512512 3002 ts_manager.cc:194] Registered new tserver with Master: d112e9fd3d5f4a12b65bc089943095a9 (127.17.69.67:41233)
I20260812 06:52:40.522276 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 06:52:40.528115 3056 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:52:40.526643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.17.69.126:39755)
0812 06:52:40.526811 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.526817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.527388 (+ 571us) server_negotiation.cc:407] Connection header received
0812 06:52:40.527496 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.527499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.527547 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.527615 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.527880 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56226: BlockingRecv error: recv got EOF from 127.0.0.1:56226 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:52:40.528834 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 3092
I20260812 06:52:40.531909 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.529674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.17.69.65:37017)
0812 06:52:40.530169 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.530176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.530199 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:52:40.530251 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.530256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.530465 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.530572 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.531359 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:40.531366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:40.531376 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:40.531393 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:40.531435 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:40.531449 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:40.531548 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:40.531615 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 06:52:40.531687 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260812 06:52:40.532944 3313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:40.615828 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 3092
I20260812 06:52:40.639398 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 3332
I20260812 06:52:40.642912 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.640285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.17.69.66:33439)
0812 06:52:40.640788 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.640796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.640821 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:52:40.640884 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.640889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.641105 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.641250 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.642178 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:40.642185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:40.642196 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:40.642214 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:40.642252 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:40.642265 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:40.642406 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:40.642537 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 06:52:40.642651 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":234,"threads_started":1}
I20260812 06:52:40.644042 3562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:40.723052 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 3332
I20260812 06:52:40.743780 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 3576
I20260812 06:52:40.747174 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:40.744586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.17.69.67:41233)
0812 06:52:40.745217 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:40.745224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:40.745257 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:52:40.745320 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:40.745327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:40.745537 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:40.745665 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:40.746543 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:40.746550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:40.746561 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:52:40.746574 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:40.746619 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:52:40.746636 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:40.746764 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:40.746811 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 06:52:40.746897 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":275,"threads_started":1}
I20260812 06:52:40.748379 3811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:40.827131 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 3576
I20260812 06:52:40.846192 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 2954
I20260812 06:52:40.847385 3044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:40.945051 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 2954
2026-08-12T06:52:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2253 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:52:41 dist-test-slave-6zbq krb5kdc[3861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:52:41 dist-test-slave-6zbq krb5kdc[3861](info): set up 2 sockets
Aug 12 06:52:41 dist-test-slave-6zbq krb5kdc[3861](info): commencing operation
krb5kdc: starting...
W20260812 06:52:42.196748 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:52:42 dist-test-slave-6zbq krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:52:42.479140 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38773
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:43.157262 3885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:43.157696 3885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:43.157934 3885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:43.169715 3885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:52:43.169889 3885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:43.170023 3885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:43.170149 3885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:52:43.170244 3885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:52:43.188491 3885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:43.190721 3885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:43.193059 3885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:43.213344 3898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:43.213376 3899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:43.213546 3901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:43.215212 3885 server_base.cc:1034] running on GCE node
I20260812 06:52:43.216647 3885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:43.219141 3885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:43.220422 3885 hybrid_clock.cc:648] HybridClock initialized: now 1786517563220352 us; error 64 us; skew 500 ppm
I20260812 06:52:43.221026 3885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:43 dist-test-slave-6zbq krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517563, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:43.230497 3885 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:52:43.233044 3885 webserver.cc:460] Webserver started at http://127.17.69.126:36135/ using document root <none> and password file <none>
I20260812 06:52:43.234301 3885 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:43.234539 3885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:43.235069 3885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:43.240329 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "2f86091f8c284d37851158b9beb1b50b"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "8b465eecbbb900b46fb62e66a9551da4"
server_key_iv: "5bf70144f09015930e10a78f7c66ddb8"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.241746 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "2f86091f8c284d37851158b9beb1b50b"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "8b465eecbbb900b46fb62e66a9551da4"
server_key_iv: "5bf70144f09015930e10a78f7c66ddb8"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.250202 3885 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260812 06:52:43.256829 3915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:43.259115 3885 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260812 06:52:43.259368 3885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "2f86091f8c284d37851158b9beb1b50b"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "8b465eecbbb900b46fb62e66a9551da4"
server_key_iv: "5bf70144f09015930e10a78f7c66ddb8"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.259829 3885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:43.313844 3885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:43.317886 3885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:43.318145 3885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:43.335161 3885 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38773
I20260812 06:52:43.335182 4005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38773 every 8 connection(s)
I20260812 06:52:43.336434 3885 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:52:43.340036 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 3885
I20260812 06:52:43.340155 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:43.340262 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:52:43.341089 17685 external_mini_cluster.cc:1428] Setting key a16c74c691932a9e459c044c837f378e
I20260812 06:52:43.351960 4006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: Bootstrap starting.
Aug 12 06:52:43 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:43.355062 4006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:43.356051 4006 log.cc:824] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:43.359262 4006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: No bootstrap required, opened a new log
I20260812 06:52:43.363823 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:43.343247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.17.69.126:38773)
0812 06:52:43.343973 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:43.343984 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:43.344021 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:52:43.346273 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:43.346293 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:43.346689 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:43.347033 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:43.348440 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.349706 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.351041 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.351652 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.351823 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:43.355851 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:43.355860 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:43.355873 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:43.355918 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:43.359470 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:43.360149 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:43.360158 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:43.360165 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:43.360391 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:43.360719 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:43.360724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:43.360727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:43.361433 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:43.361773 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:43.362054 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 06:52:43.362536 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":311,"threads_started":1}
I20260812 06:52:43.369158 4006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } }
I20260812 06:52:43.369563 4006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:43.369666 4006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f86091f8c284d37851158b9beb1b50b, State: Initialized, Role: FOLLOWER
I20260812 06:52:43.370221 4006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } }
I20260812 06:52:43.370501 4006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:52:43.370642 4006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:52:43.370820 4006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:43.372816 4006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } }
I20260812 06:52:43.373266 4006 leader_election.cc:302] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f86091f8c284d37851158b9beb1b50b; no voters:
I20260812 06:52:43.373768 4006 leader_election.cc:288] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:52:43.373871 4014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:43.374622 4014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [term 1 LEADER]: Becoming Leader. State: Replica: 2f86091f8c284d37851158b9beb1b50b, State: Running, Role: LEADER
I20260812 06:52:43.375133 4014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } }
I20260812 06:52:43.375757 4006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:52:43.377931 4016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f86091f8c284d37851158b9beb1b50b. Latest consensus state: current_term: 1 leader_uuid: "2f86091f8c284d37851158b9beb1b50b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } } }
I20260812 06:52:43.378535 4016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:43.378569 4015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f86091f8c284d37851158b9beb1b50b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f86091f8c284d37851158b9beb1b50b" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38773 } } }
I20260812 06:52:43.378929 4015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b [sys.catalog]: This master's current role is: LEADER
I20260812 06:52:43.383068 4034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:52:43.388062 4034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:52:43.396207 4034 catalog_manager.cc:1347] Generated new cluster ID: affe407c88d1402b8d488b0eb999b584
I20260812 06:52:43.396310 4034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:52:43.417426 4034 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:52:43.419155 4034 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:52:43.431170 4034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: Generated new TSK 0
I20260812 06:52:43.432060 4034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:52:43.496857 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:43.818013 4062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:43.818276 4062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:43.818428 4062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:43.825788 4062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:43.825888 4062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:43.826040 4062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:52:43.835270 4062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:43.836431 4062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:43.837635 4062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:43.850028 4073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:43.849987 4074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:43.850052 4077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:43.850871 4062 server_base.cc:1034] running on GCE node
I20260812 06:52:43.851423 4062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:43.852263 4062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:43.853461 4062 hybrid_clock.cc:648] HybridClock initialized: now 1786517563853422 us; error 26 us; skew 500 ppm
I20260812 06:52:43.853735 4062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:43 dist-test-slave-6zbq krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517563, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:43.856637 4062 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:52:43.857966 4062 webserver.cc:460] Webserver started at http://127.17.69.65:43857/ using document root <none> and password file <none>
I20260812 06:52:43.858620 4062 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:43.858723 4062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:43.858966 4062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:43.861701 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "7a25e39b28a84a8383ef0e19821400ac"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "056abb0bedbb18743ec54f7d89864c6f"
server_key_iv: "8cd158da9b0a3415b0ef7d28d758480b"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.862422 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "7a25e39b28a84a8383ef0e19821400ac"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "056abb0bedbb18743ec54f7d89864c6f"
server_key_iv: "8cd158da9b0a3415b0ef7d28d758480b"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.867260 4062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:52:43.870519 4092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:43.871753 4062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:52:43.871901 4062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "7a25e39b28a84a8383ef0e19821400ac"
format_stamp: "Formatted at 2026-08-12 06:52:43 on dist-test-slave-6zbq"
server_key: "056abb0bedbb18743ec54f7d89864c6f"
server_key_iv: "8cd158da9b0a3415b0ef7d28d758480b"
server_key_version: "encryptionkey@0"
I20260812 06:52:43.872069 4062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:43.884740 4062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:43.888324 4062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:43.888556 4062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:43.889467 4062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:43.890694 4062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:43.890774 4062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:43.890856 4062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:43.890901 4062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:43.918112 4062 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:43829
I20260812 06:52:43.918169 4271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:43829 every 8 connection(s)
I20260812 06:52:43.919266 4062 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:52:43.927591 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 4062
I20260812 06:52:43.927767 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:52:43.928372 17685 external_mini_cluster.cc:1428] Setting key 2f409121c791325e14ef6557a3ac6645
Aug 12 06:52:43 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517563, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:43.938184 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:43.921618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:38975 (local address 127.17.69.126:38773)
0812 06:52:43.922052 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:43.922056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:43.922925 (+ 869us) server_negotiation.cc:407] Connection header received
0812 06:52:43.924651 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:43.924657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:43.924736 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:43.924834 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:43.926216 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.926994 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.928544 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.929117 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.929219 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:43.933024 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:43.933030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:43.933033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:43.933066 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:43.935398 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:43.936152 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:43.936155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:43.936158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:43.936280 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:43.936687 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:43.936690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:43.936692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:43.936954 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:43.937061 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:43.937760 (+ 699us) server_negotiation.cc:299] Negotiation successful
0812 06:52:43.937942 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":222,"threads_started":1}
I20260812 06:52:43.939333 4277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:43.922075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38773 (local address 127.17.69.65:38975)
0812 06:52:43.922779 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:43.922814 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:43.924397 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:43.925062 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:43.925071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:43.925532 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:43.926050 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:43.926066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.927150 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.927154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:43.928404 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:43.928417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:43.929347 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:43.929355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:43.929577 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:43.930235 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:43.930267 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:43.932814 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:43.935539 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:43.935547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:43.935559 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:43.936049 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:43.936370 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:43.936374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:43.936377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:43.936602 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:43.937062 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:43.937069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:43.937433 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 06:52:43.937689 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 06:52:43.938142 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":294,"threads_started":1}
I20260812 06:52:43.940678 4272 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38773
I20260812 06:52:43.940963 4272 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:43.941759 4272 heartbeater.cc:511] Master 127.17.69.126:38773 requested a full tablet report, sending...
I20260812 06:52:43.943735 3944 ts_manager.cc:194] Registered new tserver with Master: 7a25e39b28a84a8383ef0e19821400ac (127.17.69.65:43829)
I20260812 06:52:43.945128 3944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:38975
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:52:43.987099 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:44.304131 4281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:44.304400 4281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:44.304522 4281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:44.311504 4281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:44.311623 4281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:44.311776 4281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:52:44.321061 4281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:44.322263 4281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:44.323483 4281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:44.335717 4289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:44.335881 4292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:44.336051 4288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:44.336578 4281 server_base.cc:1034] running on GCE node
I20260812 06:52:44.337234 4281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:44.338208 4281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:44.339445 4281 hybrid_clock.cc:648] HybridClock initialized: now 1786517564339358 us; error 70 us; skew 500 ppm
I20260812 06:52:44.339774 4281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:44 dist-test-slave-6zbq krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:44.343262 4281 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:52:44.351143 4281 webserver.cc:460] Webserver started at http://127.17.69.66:35813/ using document root <none> and password file <none>
I20260812 06:52:44.351742 4281 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:44.351847 4281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:44.352105 4281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:44.354769 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "02b8101c2f624362943f4d2451446c33"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "c273758b10e82fc2b1a287fd4d6dc70f"
server_key_iv: "b7d9e137137c17ab89fac8cf3af8404c"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.355461 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "02b8101c2f624362943f4d2451446c33"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "c273758b10e82fc2b1a287fd4d6dc70f"
server_key_iv: "b7d9e137137c17ab89fac8cf3af8404c"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.360059 4281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:52:44.363140 4305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.364295 4281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260812 06:52:44.364434 4281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "02b8101c2f624362943f4d2451446c33"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "c273758b10e82fc2b1a287fd4d6dc70f"
server_key_iv: "b7d9e137137c17ab89fac8cf3af8404c"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.364601 4281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:44.393841 4281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:44.397576 4281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:44.397832 4281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:44.398741 4281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:44.399930 4281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:44.400000 4281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.400107 4281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:44.400163 4281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.428334 4281 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:43067
I20260812 06:52:44.428390 4517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:43067 every 8 connection(s)
I20260812 06:52:44.429458 4281 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:52:44.438917 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 4281
I20260812 06:52:44.439085 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:52:44.439534 17685 external_mini_cluster.cc:1428] Setting key e8595fa13ac205e89b88add76747ed25
Aug 12 06:52:44 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:44.448006 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.431693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:34081 (local address 127.17.69.126:38773)
0812 06:52:44.431860 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.431864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.432849 (+ 985us) server_negotiation.cc:407] Connection header received
0812 06:52:44.434633 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.434639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.434714 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.434800 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:44.436149 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.436850 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.438310 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.438753 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.438865 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.442687 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:44.442692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:44.442696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:44.442726 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:44.445141 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.445974 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.445980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.445983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.446122 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.446566 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.446568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.446571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.446814 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:44.446922 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:44.447596 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 06:52:44.447777 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:52:44.449111 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.432059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38773 (local address 127.17.69.66:34081)
0812 06:52:44.432709 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:44.432746 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:44.434326 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:44.435030 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:44.435038 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:44.435491 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:44.435989 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.436005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.436994 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.436998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.438176 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.438188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.438961 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.438968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.439228 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.439847 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:44.439875 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:44.442487 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:44.445286 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:44.445294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:44.445306 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:44.445802 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:44.446221 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:44.446226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:44.446229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:44.446475 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:44.446923 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:44.446930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:44.447277 (+ 347us) client_negotiation.cc:769] Sending connection context
0812 06:52:44.447530 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 06:52:44.447984 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":272,"threads_started":1}
I20260812 06:52:44.450434 4518 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38773
I20260812 06:52:44.450704 4518 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:44.451339 4518 heartbeater.cc:511] Master 127.17.69.126:38773 requested a full tablet report, sending...
I20260812 06:52:44.452617 3944 ts_manager.cc:194] Registered new tserver with Master: 02b8101c2f624362943f4d2451446c33 (127.17.69.66:43067)
I20260812 06:52:44.453378 3944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:34081
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:52:44.497005 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:43483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:52:44.810182 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:52:44.810478 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:52:44.810618 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:52:44.817492 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:52:44.817586 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:52:44.817731 4525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:52:44.826889 4525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:43483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:52:44.828027 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:52:44.829249 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:52:44.841411 4534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:44.841451 4539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:52:44.841472 4537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:52:44.842108 4525 server_base.cc:1034] running on GCE node
I20260812 06:52:44.842731 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:52:44.843580 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:52:44.844775 4525 hybrid_clock.cc:648] HybridClock initialized: now 1786517564844724 us; error 45 us; skew 500 ppm
I20260812 06:52:44.845050 4525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:52:44 dist-test-slave-6zbq krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:52:44.848037 4525 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:52:44.849411 4525 webserver.cc:460] Webserver started at http://127.17.69.67:33575/ using document root <none> and password file <none>
I20260812 06:52:44.850049 4525 fs_manager.cc:359] Metadata directory not provided
I20260812 06:52:44.850160 4525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:52:44.850490 4525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:52:44.853147 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "f95cc31153314f52a9b40d173065de87"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "3e866111e4778cc46e1bd3694e3745b2"
server_key_iv: "a89bd891777fb6f14ffd2d7e60020bcc"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.853873 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "f95cc31153314f52a9b40d173065de87"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "3e866111e4778cc46e1bd3694e3745b2"
server_key_iv: "a89bd891777fb6f14ffd2d7e60020bcc"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.858538 4525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 06:52:44.861590 4549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.862892 4525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:52:44.863029 4525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "f95cc31153314f52a9b40d173065de87"
format_stamp: "Formatted at 2026-08-12 06:52:44 on dist-test-slave-6zbq"
server_key: "3e866111e4778cc46e1bd3694e3745b2"
server_key_iv: "a89bd891777fb6f14ffd2d7e60020bcc"
server_key_version: "encryptionkey@0"
I20260812 06:52:44.863201 4525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:52:44.889943 4525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:52:44.893795 4525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:52:44.894059 4525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:52:44.894924 4525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:52:44.896148 4525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:52:44.896235 4525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.896353 4525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:52:44.896404 4525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:52:44.925920 4525 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:45303
I20260812 06:52:44.925961 4747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:45303 every 8 connection(s)
I20260812 06:52:44.927057 4525 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:52:44.937355 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 4525
I20260812 06:52:44.937505 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:52:44.937906 17685 external_mini_cluster.cc:1428] Setting key 14ac4b3bce5da6ee4431f943641d6f98
Aug 12 06:52:44 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517564, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:52:44.944816 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.929221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:59053 (local address 127.17.69.126:38773)
0812 06:52:44.929377 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.929381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.930368 (+ 987us) server_negotiation.cc:407] Connection header received
0812 06:52:44.932044 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.932050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.932137 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.932224 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:44.933628 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.934424 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.935878 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.936317 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.936427 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.939909 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:44.939914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:44.939917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:44.939949 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:44.942397 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.943018 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.943021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.943023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.943141 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.943547 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.943550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.943552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.943819 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:44.943924 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:44.944440 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 06:52:44.944589 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:52:44.945649 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.929560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38773 (local address 127.17.69.67:59053)
0812 06:52:44.930197 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:44.930232 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:44.931784 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:44.932454 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:44.932461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:44.932913 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:52:44.933449 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.933466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.934566 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.934570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.935729 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.935747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.936526 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.936534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.936771 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.937355 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:52:44.937376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:52:44.939732 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:52:44.942526 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:44.942533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:44.942542 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:44.942934 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:44.943263 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:52:44.943267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:52:44.943269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:52:44.943456 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:52:44.943906 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:52:44.943911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:52:44.944169 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 06:52:44.944381 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 06:52:44.944709 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":246,"threads_started":1}
I20260812 06:52:44.946906 4748 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38773
I20260812 06:52:44.947160 4748 heartbeater.cc:464] Registering TS with master...
I20260812 06:52:44.947818 4272 heartbeater.cc:503] Master 127.17.69.126:38773 was elected leader, sending a full tablet report...
I20260812 06:52:44.947871 4748 heartbeater.cc:511] Master 127.17.69.126:38773 requested a full tablet report, sending...
I20260812 06:52:44.949229 3943 ts_manager.cc:194] Registered new tserver with Master: f95cc31153314f52a9b40d173065de87 (127.17.69.67:45303)
I20260812 06:52:44.950031 3943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:59053
I20260812 06:52:44.952708 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:52:44.967960 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.956809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.17.69.126:38773)
0812 06:52:44.956988 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.956994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.957345 (+ 351us) server_negotiation.cc:407] Connection header received
0812 06:52:44.957459 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.957463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.957515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.957585 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:44.958925 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.959618 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.960727 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.961125 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.961208 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.963284 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:44.963289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:44.963292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:44.963319 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:44.965383 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.965986 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.965991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.965994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.966118 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:44.966515 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:44.966519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:44.966521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:44.966834 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:44.967051 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:44.967502 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 06:52:44.967698 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:52:44.971676 3943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:52:44.978291 3943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:52:44.999629 4776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.989559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:45303 (local address 127.0.0.1:51780)
0812 06:52:44.991420 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:44.991442 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:44.991551 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:44.992635 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:44.992641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:44.992670 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:44.993051 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.993064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.994878 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.994883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.996999 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.997011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.998659 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.998666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.998856 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.998905 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 06:52:44.999036 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 06:52:44.999216 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"mutex_wait_us":45,"thread_start_us":2189,"threads_started":1}
I20260812 06:52:44.999882 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.989324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.17.69.66:43067)
0812 06:52:44.992585 (+ 3261us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.992593 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.992617 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:52:44.992709 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.992716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.992916 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.993104 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:44.993696 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.994982 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.997441 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.998854 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.998976 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.999148 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:44.999411 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 06:52:44.999566 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3117,"thread_start_us":2979,"threads_started":1}
I20260812 06:52:45.000476 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.990857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:43829 (local address 127.0.0.1:44064)
0812 06:52:44.992001 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:44.992023 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:44.992115 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:44.994248 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:44.994253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:44.994278 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:44.994619 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.994630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.996420 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.996425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.998069 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.998081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.999727 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.999735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.999884 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.999914 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:52:44.999980 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 06:52:45.000091 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":2,"thread_start_us":304,"threads_started":1}
I20260812 06:52:45.000684 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.989161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:43067 (local address 127.0.0.1:37766)
0812 06:52:44.990477 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:44.990528 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:44.990673 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:44.993105 (+ 2432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:44.993110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:44.993138 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:44.993506 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.993515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.995122 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.995128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.997288 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:44.997303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.999054 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.999061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:44.999222 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.999256 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:52:44.999335 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:52:44.999452 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":2,"thread_start_us":937,"threads_started":1}
I20260812 06:52:45.001483 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.991229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.17.69.65:43829)
0812 06:52:44.993757 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.993765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.993790 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:52:44.993872 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.993879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.994064 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.994248 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:44.994795 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.996264 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.998239 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.999507 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.000698 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.000922 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.000987 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.001150 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2355,"thread_start_us":2207,"threads_started":1}
I20260812 06:52:45.001768 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:44.990112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.17.69.67:45303)
0812 06:52:44.991767 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:44.991775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:44.991809 (+ 34us) server_negotiation.cc:407] Connection header received
0812 06:52:44.991900 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:44.991908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:44.992136 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:44.992314 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:44.993252 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.994769 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.997153 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:44.998472 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:44.998606 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:44.998816 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.000187 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.000345 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1483,"thread_start_us":1321,"threads_started":1}
I20260812 06:52:45.004535 4646 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 (DEFAULT_TABLE table=test-table [id=851501ebecac4e8a8db6df316e32fdd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:45.004767 4154 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 (DEFAULT_TABLE table=test-table [id=851501ebecac4e8a8db6df316e32fdd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:45.004760 4387 tablet_service.cc:1467] Processing CreateTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 (DEFAULT_TABLE table=test-table [id=851501ebecac4e8a8db6df316e32fdd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:52:45.005949 4646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe0df654d964dc3b47a4398c6fe9223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:45.006033 4154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe0df654d964dc3b47a4398c6fe9223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:45.006032 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fe0df654d964dc3b47a4398c6fe9223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:52:45.016430 4780 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Bootstrap starting.
I20260812 06:52:45.019452 4781 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Bootstrap starting.
I20260812 06:52:45.019925 4780 tablet_bootstrap.cc:654] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:45.020771 4784 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Bootstrap starting.
I20260812 06:52:45.021337 4780 log.cc:824] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:45.022651 4781 tablet_bootstrap.cc:654] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:45.023589 4784 tablet_bootstrap.cc:654] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Neither blocks nor log segments found. Creating new log.
I20260812 06:52:45.023797 4781 log.cc:824] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:45.024564 4780 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: No bootstrap required, opened a new log
I20260812 06:52:45.024595 4784 log.cc:824] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Log is configured to *not* fsync() on all Append() calls
I20260812 06:52:45.024799 4780 ts_tablet_manager.cc:1397] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 06:52:45.026736 4781 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: No bootstrap required, opened a new log
I20260812 06:52:45.027035 4781 ts_tablet_manager.cc:1397] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260812 06:52:45.027269 4784 tablet_bootstrap.cc:492] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: No bootstrap required, opened a new log
I20260812 06:52:45.027491 4784 ts_tablet_manager.cc:1397] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260812 06:52:45.033895 4784 raft_consensus.cc:348] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.034210 4784 raft_consensus.cc:374] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:45.034315 4784 raft_consensus.cc:729] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a25e39b28a84a8383ef0e19821400ac, State: Initialized, Role: FOLLOWER
I20260812 06:52:45.034880 4784 consensus_queue.cc:260] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.034737 4780 raft_consensus.cc:348] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.035188 4780 raft_consensus.cc:374] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:45.035288 4780 raft_consensus.cc:729] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95cc31153314f52a9b40d173065de87, State: Initialized, Role: FOLLOWER
I20260812 06:52:45.035975 4780 consensus_queue.cc:260] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.036577 4784 ts_tablet_manager.cc:1428] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 06:52:45.037724 4748 heartbeater.cc:503] Master 127.17.69.126:38773 was elected leader, sending a full tablet report...
I20260812 06:52:45.038525 4781 raft_consensus.cc:348] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.038838 4781 raft_consensus.cc:374] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:52:45.038941 4781 raft_consensus.cc:729] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02b8101c2f624362943f4d2451446c33, State: Initialized, Role: FOLLOWER
I20260812 06:52:45.039463 4781 consensus_queue.cc:260] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.040778 4780 ts_tablet_manager.cc:1428] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Time spent starting tablet: real 0.016s user 0.005s sys 0.010s
I20260812 06:52:45.041659 4518 heartbeater.cc:503] Master 127.17.69.126:38773 was elected leader, sending a full tablet report...
I20260812 06:52:45.042239 4781 ts_tablet_manager.cc:1428] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260812 06:52:45.171411 4275 tablet.cc:2369] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:52:45.178529 4749 tablet.cc:2369] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:52:45.181187 4519 tablet.cc:2369] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:52:45.397552 4796 raft_consensus.cc:482] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:45.397835 4796 raft_consensus.cc:504] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.399515 4796 leader_election.cc:288] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a25e39b28a84a8383ef0e19821400ac (127.17.69.65:43829), f95cc31153314f52a9b40d173065de87 (127.17.69.67:45303)
I20260812 06:52:45.403910 4793 raft_consensus.cc:482] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:52:45.404181 4793 raft_consensus.cc:504] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.405802 4793 leader_election.cc:288] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02b8101c2f624362943f4d2451446c33 (127.17.69.66:43067), f95cc31153314f52a9b40d173065de87 (127.17.69.67:45303)
I20260812 06:52:45.406366 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.399927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58013 (local address 127.17.69.67:45303)
0812 06:52:45.400176 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.400183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.400817 (+ 634us) server_negotiation.cc:407] Connection header received
0812 06:52:45.401002 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.401006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.401065 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.401156 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:45.401689 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.402405 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.404245 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.405358 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.405517 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.405590 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.405952 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.406082 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 06:52:45.406807 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.399933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:53437 (local address 127.17.69.65:43829)
0812 06:52:45.400175 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.400182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.400921 (+ 739us) server_negotiation.cc:407] Connection header received
0812 06:52:45.401097 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.401101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.401159 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.401231 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:45.401836 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.402566 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.404469 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.405694 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.405828 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.405896 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.406439 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.406571 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:52:45.407759 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.399861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:45303 (local address 127.17.69.66:58013)
0812 06:52:45.400716 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:45.400730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:45.400871 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:45.401185 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:45.401190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:45.401214 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:45.401543 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.401552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.402520 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.402525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.404088 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.404111 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.405584 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.405590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.405745 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.405780 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:52:45.407191 (+ 1411us) client_negotiation.cc:240] Negotiation successful
0812 06:52:45.407351 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":2,"thread_start_us":516,"threads_started":1}
I20260812 06:52:45.408325 4800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.399900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:43829 (local address 127.17.69.66:53437)
0812 06:52:45.400817 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:45.400831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:45.400956 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:45.401279 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:45.401283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:45.401322 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:45.401690 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.401700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.402702 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.402706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.404293 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.404308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.406037 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.406044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.406224 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.406257 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:52:45.407194 (+ 937us) client_negotiation.cc:240] Negotiation successful
0812 06:52:45.407741 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":3,"thread_start_us":493,"threads_started":1}
I20260812 06:52:45.409314 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "02b8101c2f624362943f4d2451446c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95cc31153314f52a9b40d173065de87" is_pre_election: true
I20260812 06:52:45.409461 4182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "02b8101c2f624362943f4d2451446c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a25e39b28a84a8383ef0e19821400ac" is_pre_election: true
I20260812 06:52:45.409869 4677 raft_consensus.cc:2454] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b8101c2f624362943f4d2451446c33 in term 0.
I20260812 06:52:45.409924 4182 raft_consensus.cc:2454] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b8101c2f624362943f4d2451446c33 in term 0.
I20260812 06:52:45.410586 4308 leader_election.cc:302] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b8101c2f624362943f4d2451446c33, 7a25e39b28a84a8383ef0e19821400ac; no voters:
I20260812 06:52:45.410971 4796 raft_consensus.cc:2781] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:52:45.411142 4796 raft_consensus.cc:482] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:52:45.411255 4796 raft_consensus.cc:3037] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:45.414422 4796 raft_consensus.cc:504] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.414886 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.406750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:57097 (local address 127.17.69.66:43067)
0812 06:52:45.406921 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.406927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.408563 (+ 1636us) server_negotiation.cc:407] Connection header received
0812 06:52:45.408978 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.408984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.409046 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.409102 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:45.409908 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.410931 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.413189 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.413994 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.414124 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.414187 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.414534 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.414649 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:52:45.414956 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.406314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:43067 (local address 127.17.69.65:57097)
0812 06:52:45.408490 (+ 2176us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:45.408504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:45.408834 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:45.409270 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:45.409275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:45.409302 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:45.409733 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.409748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.411105 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.411109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.413044 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.413060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.414211 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.414218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.414381 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.414415 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:52:45.414497 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 06:52:45.414618 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2070,"thread_start_us":1913,"threads_started":1}
I20260812 06:52:45.415437 4182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "02b8101c2f624362943f4d2451446c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a25e39b28a84a8383ef0e19821400ac"
I20260812 06:52:45.415596 4182 raft_consensus.cc:3037] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:45.415838 4796 leader_election.cc:288] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [CANDIDATE]: Term 1 election: Requested vote from peers 7a25e39b28a84a8383ef0e19821400ac (127.17.69.65:43829), f95cc31153314f52a9b40d173065de87 (127.17.69.67:45303)
I20260812 06:52:45.415920 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "7a25e39b28a84a8383ef0e19821400ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02b8101c2f624362943f4d2451446c33" is_pre_election: true
I20260812 06:52:45.416353 4438 raft_consensus.cc:2379] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a25e39b28a84a8383ef0e19821400ac in current term 1: Already voted for candidate 02b8101c2f624362943f4d2451446c33 in this term.
I20260812 06:52:45.416626 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "02b8101c2f624362943f4d2451446c33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95cc31153314f52a9b40d173065de87"
I20260812 06:52:45.416819 4677 raft_consensus.cc:3037] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:52:45.417552 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.406670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:42419 (local address 127.17.69.67:45303)
0812 06:52:45.407153 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.407159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.408371 (+ 1212us) server_negotiation.cc:407] Connection header received
0812 06:52:45.408623 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.408628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.408703 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.408761 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:52:45.409570 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.410756 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.415317 (+ 4561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.416454 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.416615 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.416680 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.417220 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.417356 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":308,"server-negotiator.queue_time_us":385}
I20260812 06:52:45.418073 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.406174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:45303 (local address 127.17.69.65:42419)
0812 06:52:45.408266 (+ 2092us) negotiation.cc:107] Waiting for socket to connect
0812 06:52:45.408284 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:52:45.408471 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:52:45.408922 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:52:45.408926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:52:45.408949 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:52:45.409402 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.409414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.413221 (+ 3807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.413225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.415130 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:52:45.415146 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.416849 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.416856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:52:45.417031 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.417067 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:52:45.417690 (+ 623us) client_negotiation.cc:240] Negotiation successful
0812 06:52:45.417825 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1958,"thread_start_us":1754,"threads_started":1}
I20260812 06:52:45.418746 4182 raft_consensus.cc:2454] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b8101c2f624362943f4d2451446c33 in term 1.
I20260812 06:52:45.418768 4676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fe0df654d964dc3b47a4398c6fe9223" candidate_uuid: "7a25e39b28a84a8383ef0e19821400ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95cc31153314f52a9b40d173065de87" is_pre_election: true
I20260812 06:52:45.419178 4308 leader_election.cc:302] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b8101c2f624362943f4d2451446c33, 7a25e39b28a84a8383ef0e19821400ac; no voters:
I20260812 06:52:45.419456 4796 raft_consensus.cc:2781] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:52:45.419567 4096 leader_election.cc:302] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a25e39b28a84a8383ef0e19821400ac; no voters: 02b8101c2f624362943f4d2451446c33, f95cc31153314f52a9b40d173065de87
I20260812 06:52:45.419885 4793 raft_consensus.cc:2726] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 06:52:45.419946 4677 raft_consensus.cc:2454] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b8101c2f624362943f4d2451446c33 in term 1.
I20260812 06:52:45.420287 4796 raft_consensus.cc:686] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 LEADER]: Becoming Leader. State: Replica: 02b8101c2f624362943f4d2451446c33, State: Running, Role: LEADER
I20260812 06:52:45.420723 4796 consensus_queue.cc:237] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } }
I20260812 06:52:45.424649 3943 catalog_manager.cc:5495] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02b8101c2f624362943f4d2451446c33 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "02b8101c2f624362943f4d2451446c33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b8101c2f624362943f4d2451446c33" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 } health_report { overall_health: UNKNOWN } } }
I20260812 06:52:45.447721 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.443008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.17.69.66:43067)
0812 06:52:45.443137 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.443141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.443359 (+ 218us) server_negotiation.cc:407] Connection header received
0812 06:52:45.443508 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.443511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.443582 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.443698 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:52:45.443970 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.444642 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.445811 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.446168 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.446252 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.446758 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:52:45.447275 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:52:45.447382 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.447476 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.447532 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 06:52:45.454279 4677 raft_consensus.cc:1261] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 1 FOLLOWER]: Refusing update from remote peer 02b8101c2f624362943f4d2451446c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:45.454433 4182 raft_consensus.cc:1261] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 1 FOLLOWER]: Refusing update from remote peer 02b8101c2f624362943f4d2451446c33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:52:45.455063 4803 consensus_queue.cc:1035] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f95cc31153314f52a9b40d173065de87" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:45.455364 4796 consensus_queue.cc:1035] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a25e39b28a84a8383ef0e19821400ac" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:52:45.464893 4808 mvcc.cc:205] Tried to move back new op lower bound from 7317575948093272064 to 7317575947966578688. Current Snapshot: MvccSnapshot[applied={T|T < 7317575948093272064}]
I20260812 06:52:45.470146 4809 mvcc.cc:205] Tried to move back new op lower bound from 7317575948093272064 to 7317575947966578688. Current Snapshot: MvccSnapshot[applied={T|T < 7317575948093272064}]
I20260812 06:52:45.477429 3944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:45.477674 3944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:52:45.481009 3944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f86091f8c284d37851158b9beb1b50b: Sending DeleteTablet for 3 replicas of tablet 6fe0df654d964dc3b47a4398c6fe9223
I20260812 06:52:45.482085 4154 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:45 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:44064
I20260812 06:52:45.482088 4387 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:45 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 06:52:45.482517 4646 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fe0df654d964dc3b47a4398c6fe9223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:52:45 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:51780
I20260812 06:52:45.483781 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 4062
I20260812 06:52:45.497237 4821 tablet_replica.cc:330] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: stopping tablet replica
I20260812 06:52:45.497527 4821 raft_consensus.cc:2229] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:52:45.497807 4822 tablet_replica.cc:330] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: stopping tablet replica
I20260812 06:52:45.498047 4821 raft_consensus.cc:2258] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:45.498164 4822 raft_consensus.cc:2229] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:45.500008 4821 ts_tablet_manager.cc:1905] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:45.501503 4819 tablet_replica.cc:330] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: stopping tablet replica
I20260812 06:52:45.501848 4819 raft_consensus.cc:2229] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:52:45.502269 4819 raft_consensus.cc:2258] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:52:45.504081 4819 ts_tablet_manager.cc:1905] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:45.504703 4822 raft_consensus.cc:2258] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 12 06:52:45 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:52:45.508049 4821 ts_tablet_manager.cc:1918] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:45.508251 4821 log.cc:1196] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/6fe0df654d964dc3b47a4398c6fe9223
I20260812 06:52:45.508821 4821 ts_tablet_manager.cc:1939] T 6fe0df654d964dc3b47a4398c6fe9223 P 02b8101c2f624362943f4d2451446c33: Deleting consensus metadata
I20260812 06:52:45.510524 3918 catalog_manager.cc:4841] TS 02b8101c2f624362943f4d2451446c33 (127.17.69.66:43067): tablet 6fe0df654d964dc3b47a4398c6fe9223 (table test-table [id=851501ebecac4e8a8db6df316e32fdd7]) successfully deleted
I20260812 06:52:45.510811 4819 ts_tablet_manager.cc:1918] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:45.510885 4822 ts_tablet_manager.cc:1905] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:52:45.511157 4819 log.cc:1196] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/6fe0df654d964dc3b47a4398c6fe9223
I20260812 06:52:45.511658 4819 ts_tablet_manager.cc:1939] T 6fe0df654d964dc3b47a4398c6fe9223 P 7a25e39b28a84a8383ef0e19821400ac: Deleting consensus metadata
I20260812 06:52:45.513171 3918 catalog_manager.cc:4841] TS 7a25e39b28a84a8383ef0e19821400ac (127.17.69.65:43829): tablet 6fe0df654d964dc3b47a4398c6fe9223 (table test-table [id=851501ebecac4e8a8db6df316e32fdd7]) successfully deleted
I20260812 06:52:45.516081 4822 ts_tablet_manager.cc:1918] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:52:45.516196 4822 log.cc:1196] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/6fe0df654d964dc3b47a4398c6fe9223
I20260812 06:52:45.516620 4822 ts_tablet_manager.cc:1939] T 6fe0df654d964dc3b47a4398c6fe9223 P f95cc31153314f52a9b40d173065de87: Deleting consensus metadata
I20260812 06:52:45.518005 3922 catalog_manager.cc:4841] TS f95cc31153314f52a9b40d173065de87 (127.17.69.67:45303): tablet 6fe0df654d964dc3b47a4398c6fe9223 (table test-table [id=851501ebecac4e8a8db6df316e32fdd7]) successfully deleted
I20260812 06:52:45.522804 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.485445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.17.69.65:43829)
0812 06:52:45.485629 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.485635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.497051 (+ 11416us) server_negotiation.cc:407] Connection header received
0812 06:52:45.498669 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.498676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.498772 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.498872 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:45.500804 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.501786 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.503243 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.503868 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.503959 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.517301 (+ 13342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:45.517314 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:45.517329 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:45.517389 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:45.520153 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.520934 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.520940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.520942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.521066 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.521473 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.521476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.521477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.521789 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:45.522013 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.522319 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.522526 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:52:45.523319 4265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:45.612082 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 4062
I20260812 06:52:45.647887 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 4281
Aug 12 06:52:45 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:52:45.663025 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.648590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.17.69.66:43067)
0812 06:52:45.648792 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.648799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.648867 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:52:45.649057 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.649062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.649135 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.649228 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:45.650440 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.651220 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.652468 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.652884 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.653001 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.657086 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:45.657098 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:45.657115 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:45.657150 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:45.660280 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.660916 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.660923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.660926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.661090 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.661449 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.661456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.661458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.661864 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:45.662166 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.662492 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.662727 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:52:45.663666 4509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:45.772662 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 4281
I20260812 06:52:45.792815 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 4525
Aug 12 06:52:45 dist-test-slave-6zbq krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:52:45.807281 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:52:45.793548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.17.69.67:45303)
0812 06:52:45.793742 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:52:45.793748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:52:45.793768 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:52:45.793913 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:52:45.793919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:52:45.794001 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:52:45.794109 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:52:45.795424 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.796110 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.797362 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:52:45.797772 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:52:45.797882 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:52:45.801989 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:52:45.802004 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:52:45.802022 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:52:45.802056 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:52:45.804706 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.805327 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.805335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.805338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.805456 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:52:45.805806 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:52:45.805810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:52:45.805812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:52:45.806151 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:52:45.806645 (+ 494us) server_negotiation.cc:1035] Waiting for connection context
0812 06:52:45.806806 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 06:52:45.807007 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:52:45.807864 4739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:45.892577 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 4525
I20260812 06:52:45.912108 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 3885
I20260812 06:52:45.913200 4000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:52:46.019356 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 3885
2026-08-12T06:52:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5075 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:52:46 dist-test-slave-6zbq krb5kdc[4840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:52:46 dist-test-slave-6zbq krb5kdc[4840](info): set up 2 sockets
Aug 12 06:52:46 dist-test-slave-6zbq krb5kdc[4840](info): commencing operation
krb5kdc: starting...
W20260812 06:52:47.252174 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:52:47 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:52:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:52:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangeradmin_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangeradmin_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangerlookup_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangerlookup_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
I20260812 06:52:47.556664 17685 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknIa6XT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/postgres -l logfile start
2026-08-12 06:52:49.705 UTC [4911] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:52:49.705 UTC [4911] LOG: listening on IPv4 address "127.17.69.84", port 55109
2026-08-12 06:52:49.708 UTC [4911] LOG: listening on Unix socket "/tmp/.s.PGSQL.55109"
2026-08-12 06:52:49.729 UTC [4916] LOG: database system was shut down at 2026-08-12 06:52:48 UTC
2026-08-12 06:52:49.732 UTC [4911] LOG: database system is ready to accept connections
I20260812 06:52:50.845135 17685 mini_postgres.cc:96] Postgres bound to 55109
2026-08-12 06:52:50.869 UTC [4927] FATAL: database "slave" does not exist
127.17.69.84:55109 - accepting connections
I20260812 06:52:50.869987 17685 mini_ranger.cc:161] Starting Ranger...
I20260812 06:52:50.896564 17685 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:52:51.270681 17685 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:52:51.270856 17685 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin
2026-08-12 06:52:51,606 [I] DB FLAVOR :POSTGRES
2026-08-12 06:52:51,606 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:52:51,606 [I] Checking connection..
2026-08-12 06:52:51,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:52:51,874 [I] Checking connection passed.
2026-08-12 06:52:51,874 [I] --------- Verifying version history table ---------
2026-08-12 06:52:51,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:52:52,118 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:52:52,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:52:52,358 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:52:52,358 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:52:52,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:52:52.589 UTC [5046] WARNING: there is no transaction in progress
2026-08-12 06:52:52,597 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:52:52,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:52:52,855 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:52:52,855 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:52:52,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:52:53,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:52:53,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-12 06:52:53,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:52:53,835 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:52:53,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:52:54,078 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:52:54,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:52:54,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:52:54,581 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:52:54,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:52:55.438 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.452 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.465 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.478 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.492 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.506 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.552 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.558 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.570 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.579 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.587 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.599 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.607 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.613 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.618 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:55.623 UTC [5242] WARNING: there is no transaction in progress
2026-08-12 06:52:56,135 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:52:56,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:52:56,380 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:52:56,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:52:56,631 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:52:56,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:52:56,881 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:52:56,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:52:57,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:52:57,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-12 06:52:57,603 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:52:57,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:52:57,823 [I] DB_PATCHES have already been applied
I20260812 06:52:57.829173 17685 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknIa6XT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:52:57.829294 17685 mini_ranger.cc:193] Using host: 127.17.69.84
I20260812 06:52:57.850426 17685 mini_ranger.cc:227] Ranger admin URL: http://127.17.69.84:44503
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:52:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:52:59.159394 17685 mini_ranger.cc:160] Time spent starting Ranger: real 8.289s user 0.003s sys 0.054s
Aug 12, 2026 6:53:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:53:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 6:53:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:53:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:53:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:53:18.892704 17685 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangerkms_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/rangerkms_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.69.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.69.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:53:19 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517599, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:53:19.132496 17685 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:53:19.158952 17685 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:53:19.844967 17685 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:53:19.845101 17685 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms
I20260812 06:53:19.845249 17685 mini_ranger_kms.cc:227] Using postgres at 127.17.69.84:55109
2026-08-12 06:53:20,364 [I] DB FLAVOR :POSTGRES
2026-08-12 06:53:20,365 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:53:20,365 [I] Checking connection
2026-08-12 06:53:20,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:53:20,595 [I] connection success
2026-08-12 06:53:20,595 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:53:20,595 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:53:20,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:53:20,818 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:53:20,818 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:53:20,818 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:53:20,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:55109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:53:21,056 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:53:21.487759 17685 mini_ranger_kms.cc:310] Created kms service
I20260812 06:53:21.651429 17685 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:53:21.692710 17685 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:53:21 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517599, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:22.081452 17685 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:53:22.082697 17685 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:53:22.082758 17685 mini_ranger_kms.cc:240] Using host: 127.17.69.84
I20260812 06:53:22.113170 17685 mini_ranger_kms.cc:276] Ranger KMS PID: 5598
I20260812 06:53:22.113307 17685 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.69.84:33001
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:53:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 06:53:24 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517604, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 06:53:24 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517604, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:24.745110 17685 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.613s user 0.093s sys 0.452s
I20260812 06:53:24.745428 17685 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:53:24.745654 17685 mini_ranger_kms.cc:401] 127.17.69.84:33001/kms/v1/keys
Aug 12 06:53:24 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517604, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
Aug 12 06:53:27 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:53:27.381211 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40745
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40745
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:33001/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:53:27.830696 5689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:53:27.831023 5689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:53:27.831204 5689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:53:27.838863 5689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:53:27.838954 5689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:53:27.839028 5689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:53:27.839107 5689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:53:27.839167 5689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:53:27.850494 5689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.69.84:33001/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40745
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:53:27.851686 5689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:53:27.853062 5689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:53:27.866267 5698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:27.866391 5697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:27.866310 5700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:53:27.867465 5689 server_base.cc:1034] running on GCE node
I20260812 06:53:27.868178 5689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:53:27.869483 5689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:53:27.870669 5689 hybrid_clock.cc:648] HybridClock initialized: now 1786517607870629 us; error 29 us; skew 500 ppm
I20260812 06:53:27.870960 5689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:53:27 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:53:27.876619 5689 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:53:27.878108 5689 webserver.cc:460] Webserver started at http://127.17.69.126:43639/ using document root <none> and password file <none>
I20260812 06:53:27.878695 5689 fs_manager.cc:359] Metadata directory not provided
I20260812 06:53:27.878801 5689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:53:27.879088 5689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:53:27 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:27.939241 5689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "2b31ec08a1d24cd4a7361791db68bd2d"
format_stamp: "Formatted at 2026-08-12 06:53:27 on dist-test-slave-6zbq"
server_key: "22f7f167d5d988a64b8b10bbc85cd0d7"
server_key_iv: "c70d17c0f01d1fc298a2696845fb468b"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:27.940073 5689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "2b31ec08a1d24cd4a7361791db68bd2d"
format_stamp: "Formatted at 2026-08-12 06:53:27 on dist-test-slave-6zbq"
server_key: "22f7f167d5d988a64b8b10bbc85cd0d7"
server_key_iv: "c70d17c0f01d1fc298a2696845fb468b"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:27.945969 5689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 06:53:27.965741 5707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:27.967268 5689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:53:27.967442 5689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "2b31ec08a1d24cd4a7361791db68bd2d"
format_stamp: "Formatted at 2026-08-12 06:53:27 on dist-test-slave-6zbq"
server_key: "22f7f167d5d988a64b8b10bbc85cd0d7"
server_key_iv: "c70d17c0f01d1fc298a2696845fb468b"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:27.967682 5689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:53:27.984323 5689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:53:27.988550 5689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:53:27.988911 5689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:53:28.017161 5689 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40745
I20260812 06:53:28.017206 5791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40745 every 8 connection(s)
I20260812 06:53:28.018656 5689 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:53:28.022779 5793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:53:28.026907 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 5689
I20260812 06:53:28.027128 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:53:28.027959 17685 external_mini_cluster.cc:1428] Setting key 08dddb4dfff3a28c61a13a91e276fafd
I20260812 06:53:28.036867 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: Bootstrap starting.
I20260812 06:53:28.040501 5793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: Neither blocks nor log segments found. Creating new log.
Aug 12 06:53:28 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:53:28.041901 5793 log.cc:824] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:53:28.045253 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: No bootstrap required, opened a new log
I20260812 06:53:28.049983 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:28.030135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.17.69.126:40745)
0812 06:53:28.031099 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:28.031111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:28.031149 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:53:28.033080 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:28.033107 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:28.033483 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:28.033835 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:28.035158 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:28.036549 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:28.037941 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:28.038798 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:28.038999 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:28.042627 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:28.042636 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:28.042653 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:28.042687 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:28.046230 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:28.046859 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:28.046865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:28.046871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:28.047016 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:28.047386 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:28.047390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:28.047392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:28.047915 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:28.048217 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:28.048651 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 06:53:28.049007 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":531,"threads_started":1}
I20260812 06:53:28.054105 5793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } }
I20260812 06:53:28.054548 5793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:53:28.054639 5793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b31ec08a1d24cd4a7361791db68bd2d, State: Initialized, Role: FOLLOWER
I20260812 06:53:28.055148 5793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } }
I20260812 06:53:28.055375 5793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:53:28.055506 5793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:53:28.055653 5793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:53:28.058305 5793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } }
I20260812 06:53:28.058879 5793 leader_election.cc:302] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b31ec08a1d24cd4a7361791db68bd2d; no voters:
I20260812 06:53:28.060096 5793 leader_election.cc:288] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:53:28.060170 5804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:53:28.061589 5804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [term 1 LEADER]: Becoming Leader. State: Replica: 2b31ec08a1d24cd4a7361791db68bd2d, State: Running, Role: LEADER
I20260812 06:53:28.062222 5804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } }
I20260812 06:53:28.062403 5793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:53:28.065009 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b31ec08a1d24cd4a7361791db68bd2d. Latest consensus state: current_term: 1 leader_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } } }
I20260812 06:53:28.064998 5805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b31ec08a1d24cd4a7361791db68bd2d" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40745 } } }
I20260812 06:53:28.065554 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [sys.catalog]: This master's current role is: LEADER
I20260812 06:53:28.065588 5805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d [sys.catalog]: This master's current role is: LEADER
I20260812 06:53:28.075974 5793 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:53:28.077111 5814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:53:28.093806 5814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:53:28.104911 5814 catalog_manager.cc:1347] Generated new cluster ID: 7f86bf370be94cf3873c6da152bebfa9
I20260812 06:53:28.105084 5814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:53:28.149120 5814 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:53:28.151083 5814 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:53:28.159675 5814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: Generated new TSK 0
I20260812 06:53:28.160674 5814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:53:28.949679 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:33001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:53:29 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:53:29.437338 5872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:53:29.437698 5872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:53:29.437906 5872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:53:29.450668 5872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:53:29.450829 5872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:53:29.451103 5872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:53:29.469080 5872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.69.84:33001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:53:29.470778 5872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:53:29.472584 5872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:53:29.491179 5893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:29.492738 5894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:53:29.498510 5872 server_base.cc:1034] running on GCE node
W20260812 06:53:29.499271 5899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:53:29.499991 5872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:53:29.501124 5872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:53:29.502419 5872 hybrid_clock.cc:648] HybridClock initialized: now 1786517609502303 us; error 102 us; skew 500 ppm
I20260812 06:53:29.502833 5872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:53:29 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:53:29.507803 5872 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:53:29.516762 5872 webserver.cc:460] Webserver started at http://127.17.69.65:43193/ using document root <none> and password file <none>
I20260812 06:53:29.517835 5872 fs_manager.cc:359] Metadata directory not provided
I20260812 06:53:29.518041 5872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:53:29.518527 5872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:53:29 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:29.542217 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "37cc9544485a4289a581fff0c8cbc6fe"
format_stamp: "Formatted at 2026-08-12 06:53:29 on dist-test-slave-6zbq"
server_key: "7027a1cf07385c674c941bef94b9d849"
server_key_iv: "772bb8e615bd2b2089b5a5bd7113fce5"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:29.545049 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "37cc9544485a4289a581fff0c8cbc6fe"
format_stamp: "Formatted at 2026-08-12 06:53:29 on dist-test-slave-6zbq"
server_key: "7027a1cf07385c674c941bef94b9d849"
server_key_iv: "772bb8e615bd2b2089b5a5bd7113fce5"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:29.559623 5872 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.012s sys 0.000s
I20260812 06:53:29.579288 5906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:29.581133 5872 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.001s
I20260812 06:53:29.581303 5872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "37cc9544485a4289a581fff0c8cbc6fe"
format_stamp: "Formatted at 2026-08-12 06:53:29 on dist-test-slave-6zbq"
server_key: "7027a1cf07385c674c941bef94b9d849"
server_key_iv: "772bb8e615bd2b2089b5a5bd7113fce5"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:29.581537 5872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:53:29.600106 5872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:53:29.609817 5872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:53:29.610261 5872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:53:29.612501 5872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:53:29.619979 5872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:53:29.620148 5872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 06:53:29.620376 5872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:53:29.620509 5872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 06:53:29 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:29.836169 5872 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:44825
I20260812 06:53:29.836229 6092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:44825 every 8 connection(s)
I20260812 06:53:29.838080 5872 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:53:29.843411 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 5872
I20260812 06:53:29.843659 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:53:29.844583 17685 external_mini_cluster.cc:1428] Setting key 5a0d8be52d12764d66be31c5be93f263
Aug 12 06:53:29 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:53:29.869939 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:29.842698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:34901 (local address 127.17.69.126:40745)
0812 06:53:29.846971 (+ 4273us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:29.846977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:29.850688 (+ 3711us) server_negotiation.cc:407] Connection header received
0812 06:53:29.850752 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:29.850757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:29.850830 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:29.850923 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:29.853029 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:29.854303 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:29.857003 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:29.857735 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:29.857832 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:29.862607 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:29.862613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:29.862618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:29.862664 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:29.866516 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:29.867370 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:29.867375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:29.867378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:29.867528 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:29.868027 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:29.868032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:29.868035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:29.868393 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:29.868503 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:29.869391 (+ 888us) server_negotiation.cc:299] Negotiation successful
0812 06:53:29.869598 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4145,"thread_start_us":4015,"threads_started":1}
I20260812 06:53:29.873111 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:29.842096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40745 (local address 127.17.69.65:34901)
0812 06:53:29.847470 (+ 5374us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:29.847516 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:29.850276 (+ 2760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:29.851322 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:29.851333 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:29.851882 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:53:29.852767 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:29.852853 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:29.854565 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:29.854571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:29.856802 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:29.856856 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:29.858247 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:29.858255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:29.858601 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:29.859347 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:53:29.859383 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:53:29.862379 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:53:29.866690 (+ 4311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:29.866699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:29.866714 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:29.867239 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:29.867680 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:29.867684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:29.867688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:29.867907 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:29.868609 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:53:29.868616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:53:29.868996 (+ 380us) client_negotiation.cc:769] Sending connection context
0812 06:53:29.870630 (+ 1634us) client_negotiation.cc:240] Negotiation successful
0812 06:53:29.871151 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5083,"thread_start_us":10979,"threads_started":1}
I20260812 06:53:29.875065 6094 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40745
I20260812 06:53:29.875408 6094 heartbeater.cc:464] Registering TS with master...
I20260812 06:53:29.876292 6094 heartbeater.cc:511] Master 127.17.69.126:40745 requested a full tablet report, sending...
I20260812 06:53:29.878931 5731 ts_manager.cc:194] Registered new tserver with Master: 37cc9544485a4289a581fff0c8cbc6fe (127.17.69.65:44825)
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
I20260812 06:53:29.880808 5731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:34901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:53:29.921051 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:33001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:53:30 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
W20260812 06:53:30.527583 6109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:53:30.528039 6109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:53:30.528374 6109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:53:30.540237 6109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:53:30.540333 6109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:53:30.540478 6109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:53:30.551510 6109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.69.84:33001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:53:30.552752 6109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:53:30.554181 6109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:53:30.566933 6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:30.567085 6136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:30.567209 6143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:53:30.568593 6109 server_base.cc:1034] running on GCE node
I20260812 06:53:30.569249 6109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:53:30.570082 6109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:53:30.571270 6109 hybrid_clock.cc:648] HybridClock initialized: now 1786517610571214 us; error 48 us; skew 500 ppm
I20260812 06:53:30.571547 6109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:53:30 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517610, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:53:30.575448 6109 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:53:30.577095 6109 webserver.cc:460] Webserver started at http://127.17.69.66:36833/ using document root <none> and password file <none>
I20260812 06:53:30.577847 6109 fs_manager.cc:359] Metadata directory not provided
I20260812 06:53:30.578003 6109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:53:30.578418 6109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:53:30 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517610, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:30.593940 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "12f64f0540384e9eb9d79f3471132926"
format_stamp: "Formatted at 2026-08-12 06:53:30 on dist-test-slave-6zbq"
server_key: "17e667f125bbba0ad8554343cc56755c"
server_key_iv: "10dbb6e2e31cd02d462323857f778eae"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:30.595067 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "12f64f0540384e9eb9d79f3471132926"
format_stamp: "Formatted at 2026-08-12 06:53:30 on dist-test-slave-6zbq"
server_key: "17e667f125bbba0ad8554343cc56755c"
server_key_iv: "10dbb6e2e31cd02d462323857f778eae"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:30.601783 6109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 06:53:30.617674 6153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:30.619380 6109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:53:30.619570 6109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "12f64f0540384e9eb9d79f3471132926"
format_stamp: "Formatted at 2026-08-12 06:53:30 on dist-test-slave-6zbq"
server_key: "17e667f125bbba0ad8554343cc56755c"
server_key_iv: "10dbb6e2e31cd02d462323857f778eae"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:30.619863 6109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:53:30.637326 6109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:53:30.642622 6109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:53:30.642941 6109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:53:30.644217 6109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:53:30.646281 6109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:53:30.646389 6109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 06:53:30.646515 6109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:53:30.646575 6109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:30.698493 6109 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:45431
I20260812 06:53:30.698513 6334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:45431 every 8 connection(s)
I20260812 06:53:30.699683 6109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:53:30.705499 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 6109
I20260812 06:53:30.705647 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:53:30.706063 17685 external_mini_cluster.cc:1428] Setting key 3dcc4ddb0f919020f27f6969e67c5f76
Aug 12 06:53:30 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517610, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:53:30.718824 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:30.702068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:42831 (local address 127.17.69.126:40745)
0812 06:53:30.702527 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:30.702531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:30.703730 (+ 1199us) server_negotiation.cc:407] Connection header received
0812 06:53:30.705881 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:30.705886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:30.705990 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:30.706089 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:30.707579 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:30.708380 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:30.709852 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:30.710249 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:30.710373 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:30.713536 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:30.713541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:30.713545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:30.713578 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:30.715986 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:30.716639 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:30.716643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:30.716646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:30.716763 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:30.717178 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:30.717182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:30.717184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:30.717449 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:30.717577 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:30.718224 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 06:53:30.718474 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":196,"threads_started":1}
I20260812 06:53:30.719722 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:30.702585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40745 (local address 127.17.69.66:42831)
0812 06:53:30.703570 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:30.703614 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:30.705529 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:30.706418 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:30.706427 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:30.706870 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:53:30.707416 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:30.707434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:30.708530 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:30.708534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:30.709710 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:30.709730 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:30.710476 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:30.710483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:30.710692 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:30.711291 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:53:30.711312 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:53:30.713335 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:53:30.716109 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:30.716118 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:30.716131 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:30.716519 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:30.716862 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:30.716867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:30.716871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:30.717057 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:30.717568 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:53:30.717575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:53:30.717847 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 06:53:30.718078 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 06:53:30.718543 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":472,"threads_started":1}
I20260812 06:53:30.720986 6336 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40745
I20260812 06:53:30.721283 6336 heartbeater.cc:464] Registering TS with master...
I20260812 06:53:30.722093 6336 heartbeater.cc:511] Master 127.17.69.126:40745 requested a full tablet report, sending...
I20260812 06:53:30.723475 5731 ts_manager.cc:194] Registered new tserver with Master: 12f64f0540384e9eb9d79f3471132926 (127.17.69.66:45431)
I20260812 06:53:30.724231 5731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:42831
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:53:30.761662 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:33001/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:53:30.884420 6094 heartbeater.cc:503] Master 127.17.69.126:40745 was elected leader, sending a full tablet report...
W20260812 06:53:31.181924 6353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:53:31.182262 6353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:53:31.182468 6353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:53:31.190450 6353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:53:31.190544 6353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:53:31.190686 6353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:53:31.200424 6353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.69.84:33001/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:53:31.201579 6353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:53:31.202960 6353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:53:31.217471 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:31.217443 6375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:53:31.217592 6365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:53:31.218747 6353 server_base.cc:1034] running on GCE node
I20260812 06:53:31.219398 6353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:53:31.220527 6353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:53:31.221774 6353 hybrid_clock.cc:648] HybridClock initialized: now 1786517611221682 us; error 34 us; skew 500 ppm
I20260812 06:53:31.222093 6353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:53:31 dist-test-slave-6zbq krb5kdc[4840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517611, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:53:31.225199 6353 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:53:31.226856 6353 webserver.cc:460] Webserver started at http://127.17.69.67:35639/ using document root <none> and password file <none>
I20260812 06:53:31.227444 6353 fs_manager.cc:359] Metadata directory not provided
I20260812 06:53:31.227579 6353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:53:31.227941 6353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:53:31 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517611, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:53:31.240685 6353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "58094ed565d04a6bae4176be6d3dd9da"
format_stamp: "Formatted at 2026-08-12 06:53:31 on dist-test-slave-6zbq"
server_key: "f853b128e809a12e1b7d4eb6bfdd8e17"
server_key_iv: "4f20169ac7d346caf94ad29107f23f62"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:31.241580 6353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "58094ed565d04a6bae4176be6d3dd9da"
format_stamp: "Formatted at 2026-08-12 06:53:31 on dist-test-slave-6zbq"
server_key: "f853b128e809a12e1b7d4eb6bfdd8e17"
server_key_iv: "4f20169ac7d346caf94ad29107f23f62"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:31.252130 6353 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.006s
I20260812 06:53:31.268545 6387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:31.270010 6353 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.004s
I20260812 06:53:31.270228 6353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "58094ed565d04a6bae4176be6d3dd9da"
format_stamp: "Formatted at 2026-08-12 06:53:31 on dist-test-slave-6zbq"
server_key: "f853b128e809a12e1b7d4eb6bfdd8e17"
server_key_iv: "4f20169ac7d346caf94ad29107f23f62"
server_key_version: "kuduclusterkey@0"
I20260812 06:53:31.270566 6353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:53:31.302251 6353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:53:31.306761 6353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:53:31.307049 6353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:53:31.308277 6353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:53:31.309789 6353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:53:31.309890 6353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:31.310047 6353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:53:31.310163 6353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:53:31.359735 6353 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:41787
I20260812 06:53:31.359804 6590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:41787 every 8 connection(s)
I20260812 06:53:31.361157 6353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:53:31.364815 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 6353
I20260812 06:53:31.365021 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:53:31.365600 17685 external_mini_cluster.cc:1428] Setting key d2799b02c2238b043157649c95f7a43d
Aug 12 06:53:31 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517611, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:53:31.381953 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.364125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:52867 (local address 127.17.69.126:40745)
0812 06:53:31.364699 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.364703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.365588 (+ 885us) server_negotiation.cc:407] Connection header received
0812 06:53:31.367763 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.367769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.367850 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.367942 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:31.369701 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.370560 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.372292 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.372743 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.372865 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.376220 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:31.376225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:31.376230 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:31.376261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:31.378927 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.379574 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.379580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.379584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.379772 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.380221 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.380225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.380227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.380529 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:31.380660 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.381281 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.381492 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":289,"threads_started":1}
I20260812 06:53:31.382843 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.364546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40745 (local address 127.17.69.67:52867)
0812 06:53:31.365438 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.365481 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.367490 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.368188 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.368199 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.368757 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:53:31.369542 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.369561 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.370701 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.370707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.372148 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.372168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.372959 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.372967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.373176 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.373726 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:53:31.373747 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:53:31.376026 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:53:31.379056 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:31.379065 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:31.379079 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:31.379465 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:31.379876 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:53:31.379881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:53:31.379885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:53:31.380105 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:53:31.380656 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:53:31.380664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:53:31.380943 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.381174 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.381720 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":406,"threads_started":1}
I20260812 06:53:31.384125 6591 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40745
I20260812 06:53:31.384449 6591 heartbeater.cc:464] Registering TS with master...
I20260812 06:53:31.385246 6591 heartbeater.cc:511] Master 127.17.69.126:40745 requested a full tablet report, sending...
I20260812 06:53:31.386559 5731 ts_manager.cc:194] Registered new tserver with Master: 58094ed565d04a6bae4176be6d3dd9da (127.17.69.67:41787)
I20260812 06:53:31.387431 5731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:52867
I20260812 06:53:31.392431 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:53:31.409150 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.397278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.17.69.126:40745)
0812 06:53:31.397469 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.397475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.397774 (+ 299us) server_negotiation.cc:407] Connection header received
0812 06:53:31.397892 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.397895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.397989 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.398063 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:31.399697 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.400437 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.401770 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.402298 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.402424 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.404848 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:31.404853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:31.404856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:31.404884 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:31.407016 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.407661 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.407664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.407666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.407806 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.408108 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.408111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.408112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.408340 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:31.408507 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.408727 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.408871 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:53:31.412981 5731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:53:31.420748 5731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:53:31.444137 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.433320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41787 (local address 127.0.0.1:42240)
0812 06:53:31.434328 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.434417 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.434586 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.435796 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.435804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.435844 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:53:31.436361 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.436374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.438735 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.438742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.441350 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.441366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.443206 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.443213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.443403 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.443455 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.443549 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.443740 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":623,"threads_started":1}
I20260812 06:53:31.445819 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.433558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.17.69.67:41787)
0812 06:53:31.435201 (+ 1643us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.435208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.435241 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:53:31.435326 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.435333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.435558 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.436559 (+ 1001us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:53:31.436921 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.438573 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.441546 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.442992 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.444855 (+ 1863us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.445098 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.445165 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.445319 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1485,"thread_start_us":1330,"threads_started":1}
I20260812 06:53:31.451642 6469 tablet_service.cc:1467] Processing CreateTablet for tablet 8347033349124342ba23e649d0019daf (DEFAULT_TABLE table=test-table [id=a91990b76b5145e594a519372fb7b24b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:53:31.453665 6469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8347033349124342ba23e649d0019daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:53:31.456135 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.433443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44825 (local address 127.0.0.1:51254)
0812 06:53:31.444361 (+ 10918us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.444380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.444475 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.448811 (+ 4336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.448815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.448841 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:53:31.448996 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.449006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.451501 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.451506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.453665 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.453681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.455429 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.455435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.455609 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.455639 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.455722 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.455854 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10825,"thread_start_us":13975,"threads_started":1}
I20260812 06:53:31.456297 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.434000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.17.69.65:44825)
0812 06:53:31.448253 (+ 14253us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.448277 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.448302 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:53:31.448393 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.448399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.448615 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.448777 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:53:31.449173 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.451343 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.453860 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.455264 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.455400 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.455625 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.455811 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.455967 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14087,"thread_start_us":13937,"threads_started":1}
I20260812 06:53:31.456635 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.434803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:45431 (local address 127.0.0.1:33736)
0812 06:53:31.446921 (+ 12118us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.446939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.447064 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.448244 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.448249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.448277 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:53:31.449577 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.449590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.451842 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.451846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.454291 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.454301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.455844 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.455850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.456018 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.456045 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.456128 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.456242 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11964,"mutex_wait_us":1,"thread_start_us":11826,"threads_started":1}
I20260812 06:53:31.457203 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.435306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.17.69.66:45431)
0812 06:53:31.443690 (+ 8384us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.443699 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.447655 (+ 3956us) server_negotiation.cc:407] Connection header received
0812 06:53:31.447763 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.447771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.448007 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.448250 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:53:31.449867 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.451718 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.454475 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.455393 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.455521 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.455711 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.456807 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.456968 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8206,"thread_start_us":8044,"threads_started":1}
I20260812 06:53:31.460065 5989 tablet_service.cc:1467] Processing CreateTablet for tablet 8347033349124342ba23e649d0019daf (DEFAULT_TABLE table=test-table [id=a91990b76b5145e594a519372fb7b24b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:53:31.460610 6238 tablet_service.cc:1467] Processing CreateTablet for tablet 8347033349124342ba23e649d0019daf (DEFAULT_TABLE table=test-table [id=a91990b76b5145e594a519372fb7b24b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:53:31.461750 5989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8347033349124342ba23e649d0019daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:53:31.462173 6238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8347033349124342ba23e649d0019daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:53:31.470861 6623 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Bootstrap starting.
I20260812 06:53:31.474582 6623 tablet_bootstrap.cc:654] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Neither blocks nor log segments found. Creating new log.
I20260812 06:53:31.476030 6623 log.cc:824] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Log is configured to *not* fsync() on all Append() calls
I20260812 06:53:31.479189 6625 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Bootstrap starting.
I20260812 06:53:31.480155 6628 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Bootstrap starting.
I20260812 06:53:31.483246 6628 tablet_bootstrap.cc:654] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Neither blocks nor log segments found. Creating new log.
I20260812 06:53:31.483584 6625 tablet_bootstrap.cc:654] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Neither blocks nor log segments found. Creating new log.
I20260812 06:53:31.484434 6628 log.cc:824] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Log is configured to *not* fsync() on all Append() calls
I20260812 06:53:31.485175 6625 log.cc:824] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Log is configured to *not* fsync() on all Append() calls
I20260812 06:53:31.489107 6623 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: No bootstrap required, opened a new log
I20260812 06:53:31.489385 6625 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: No bootstrap required, opened a new log
I20260812 06:53:31.489498 6623 ts_tablet_manager.cc:1397] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.012s
I20260812 06:53:31.489540 6628 tablet_bootstrap.cc:492] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: No bootstrap required, opened a new log
I20260812 06:53:31.489612 6625 ts_tablet_manager.cc:1397] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260812 06:53:31.489765 6628 ts_tablet_manager.cc:1397] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 06:53:31.498800 6625 raft_consensus.cc:348] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.499290 6625 raft_consensus.cc:374] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:53:31.499421 6625 raft_consensus.cc:729] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37cc9544485a4289a581fff0c8cbc6fe, State: Initialized, Role: FOLLOWER
I20260812 06:53:31.500118 6625 consensus_queue.cc:260] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.501322 6628 raft_consensus.cc:348] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.501571 6623 raft_consensus.cc:348] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.501886 6628 raft_consensus.cc:374] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:53:31.501971 6623 raft_consensus.cc:374] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:53:31.502055 6623 raft_consensus.cc:729] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58094ed565d04a6bae4176be6d3dd9da, State: Initialized, Role: FOLLOWER
I20260812 06:53:31.502025 6628 raft_consensus.cc:729] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12f64f0540384e9eb9d79f3471132926, State: Initialized, Role: FOLLOWER
I20260812 06:53:31.502650 6623 consensus_queue.cc:260] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.502846 6625 ts_tablet_manager.cc:1428] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 06:53:31.502876 6628 consensus_queue.cc:260] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.503974 6591 heartbeater.cc:503] Master 127.17.69.126:40745 was elected leader, sending a full tablet report...
I20260812 06:53:31.504667 6623 ts_tablet_manager.cc:1428] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260812 06:53:31.505939 6336 heartbeater.cc:503] Master 127.17.69.126:40745 was elected leader, sending a full tablet report...
I20260812 06:53:31.506635 6628 ts_tablet_manager.cc:1428] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Time spent starting tablet: real 0.017s user 0.017s sys 0.001s
W20260812 06:53:31.597508 6095 tablet.cc:2369] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:53:31.613852 6593 tablet.cc:2369] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:53:31.651342 6638 raft_consensus.cc:482] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:53:31.651605 6638 raft_consensus.cc:504] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.653184 6638 leader_election.cc:288] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58094ed565d04a6bae4176be6d3dd9da (127.17.69.67:41787), 37cc9544485a4289a581fff0c8cbc6fe (127.17.69.65:44825)
I20260812 06:53:31.661921 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.653628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:55875 (local address 127.17.69.65:44825)
0812 06:53:31.653836 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.653842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.654579 (+ 737us) server_negotiation.cc:407] Connection header received
0812 06:53:31.654728 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.654733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.654815 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.654917 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:53:31.655715 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.656862 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.659436 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.660650 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.660829 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.660905 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.661529 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.661664 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:53:31.661916 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.653711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:42397 (local address 127.17.69.67:41787)
0812 06:53:31.653893 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.653898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.654700 (+ 802us) server_negotiation.cc:407] Connection header received
0812 06:53:31.654776 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.654780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.654847 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.654928 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:53:31.655710 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.657055 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.659474 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.660666 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.660829 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.660905 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.661528 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.661664 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:53:31.662127 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.653556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44825 (local address 127.17.69.66:55875)
0812 06:53:31.654484 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.654497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.654596 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.654933 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.654938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.654976 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:53:31.655542 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.655556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.656973 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.656977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.659273 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.659290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.661033 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.661042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.661278 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.661363 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.661457 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.661627 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":156,"spinlock_wait_cycles":2688,"thread_start_us":448,"threads_started":1}
I20260812 06:53:31.662178 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.653419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:41787 (local address 127.17.69.66:42397)
0812 06:53:31.654417 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0812 06:53:31.654438 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:53:31.654563 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:53:31.654936 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:53:31.654940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:53:31.654982 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:53:31.655554 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.655563 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.657175 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.657181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.659278 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:53:31.659290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.661033 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.661042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:53:31.661311 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.661370 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 06:53:31.661457 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 06:53:31.661627 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":131,"spinlock_wait_cycles":5760,"thread_start_us":626,"threads_started":1}
I20260812 06:53:31.663375 6020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8347033349124342ba23e649d0019daf" candidate_uuid: "12f64f0540384e9eb9d79f3471132926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37cc9544485a4289a581fff0c8cbc6fe" is_pre_election: true
I20260812 06:53:31.663370 6504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8347033349124342ba23e649d0019daf" candidate_uuid: "12f64f0540384e9eb9d79f3471132926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58094ed565d04a6bae4176be6d3dd9da" is_pre_election: true
I20260812 06:53:31.664057 6020 raft_consensus.cc:2454] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12f64f0540384e9eb9d79f3471132926 in term 0.
I20260812 06:53:31.664103 6504 raft_consensus.cc:2454] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12f64f0540384e9eb9d79f3471132926 in term 0.
I20260812 06:53:31.664822 6155 leader_election.cc:302] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12f64f0540384e9eb9d79f3471132926, 58094ed565d04a6bae4176be6d3dd9da; no voters:
I20260812 06:53:31.665200 6638 raft_consensus.cc:2781] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:53:31.665366 6638 raft_consensus.cc:482] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:53:31.665467 6638 raft_consensus.cc:3037] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:53:31.667680 6638 raft_consensus.cc:504] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.668401 6638 leader_election.cc:288] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [CANDIDATE]: Term 1 election: Requested vote from peers 58094ed565d04a6bae4176be6d3dd9da (127.17.69.67:41787), 37cc9544485a4289a581fff0c8cbc6fe (127.17.69.65:44825)
I20260812 06:53:31.668743 6504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8347033349124342ba23e649d0019daf" candidate_uuid: "12f64f0540384e9eb9d79f3471132926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58094ed565d04a6bae4176be6d3dd9da"
I20260812 06:53:31.668897 6020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8347033349124342ba23e649d0019daf" candidate_uuid: "12f64f0540384e9eb9d79f3471132926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37cc9544485a4289a581fff0c8cbc6fe"
I20260812 06:53:31.668949 6504 raft_consensus.cc:3037] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:53:31.669101 6020 raft_consensus.cc:3037] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:53:31.672581 6020 raft_consensus.cc:2454] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12f64f0540384e9eb9d79f3471132926 in term 1.
I20260812 06:53:31.672571 6504 raft_consensus.cc:2454] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12f64f0540384e9eb9d79f3471132926 in term 1.
I20260812 06:53:31.672989 6155 leader_election.cc:302] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 12f64f0540384e9eb9d79f3471132926, 37cc9544485a4289a581fff0c8cbc6fe, 58094ed565d04a6bae4176be6d3dd9da; no voters:
I20260812 06:53:31.673385 6638 raft_consensus.cc:2781] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:53:31.674009 6638 raft_consensus.cc:686] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 1 LEADER]: Becoming Leader. State: Replica: 12f64f0540384e9eb9d79f3471132926, State: Running, Role: LEADER
I20260812 06:53:31.674698 6638 consensus_queue.cc:237] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } }
I20260812 06:53:31.679150 5726 catalog_manager.cc:5495] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12f64f0540384e9eb9d79f3471132926 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "12f64f0540384e9eb9d79f3471132926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12f64f0540384e9eb9d79f3471132926" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 45431 } health_report { overall_health: HEALTHY } } }
W20260812 06:53:31.701637 6337 tablet.cc:2369] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:53:31.749779 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.744128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.17.69.66:45431)
0812 06:53:31.744298 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.744302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.744318 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:53:31.744395 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.744398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.744476 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.744601 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:53:31.745087 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.746012 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.747422 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.748112 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.748227 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.748731 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:53:31.749170 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:53:31.749339 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.749391 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.749468 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:53:31.758615 6504 raft_consensus.cc:1261] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 1 FOLLOWER]: Refusing update from remote peer 12f64f0540384e9eb9d79f3471132926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:53:31.758661 6020 raft_consensus.cc:1261] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 1 FOLLOWER]: Refusing update from remote peer 12f64f0540384e9eb9d79f3471132926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:53:31.759747 6644 consensus_queue.cc:1035] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37cc9544485a4289a581fff0c8cbc6fe" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:53:31.760207 6638 consensus_queue.cc:1035] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58094ed565d04a6bae4176be6d3dd9da" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 41787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:53:31.770912 6655 mvcc.cc:205] Tried to move back new op lower bound from 7317576137752367104 to 7317576137423323136. Current Snapshot: MvccSnapshot[applied={T|T < 7317576137752367104}]
I20260812 06:53:31.792963 5728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:53:31.793265 5728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:53:31.797622 5728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b31ec08a1d24cd4a7361791db68bd2d: Sending DeleteTablet for 3 replicas of tablet 8347033349124342ba23e649d0019daf
I20260812 06:53:31.799005 6238 tablet_service.cc:1514] Processing DeleteTablet for tablet 8347033349124342ba23e649d0019daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:53:31 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33736
I20260812 06:53:31.799242 6469 tablet_service.cc:1514] Processing DeleteTablet for tablet 8347033349124342ba23e649d0019daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:53:31 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:42240
I20260812 06:53:31.799769 5989 tablet_service.cc:1514] Processing DeleteTablet for tablet 8347033349124342ba23e649d0019daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:53:31 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:51254
I20260812 06:53:31.800558 6668 tablet_replica.cc:330] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: stopping tablet replica
I20260812 06:53:31.801049 6668 raft_consensus.cc:2229] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:53:31.801725 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 5872
I20260812 06:53:31.801797 6668 raft_consensus.cc:2258] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:53:31.804051 6668 ts_tablet_manager.cc:1905] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:53:31.810590 6671 tablet_replica.cc:330] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: stopping tablet replica
I20260812 06:53:31.810968 6671 raft_consensus.cc:2229] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:53:31.811808 6668 ts_tablet_manager.cc:1918] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:53:31.811996 6668 log.cc:1196] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/8347033349124342ba23e649d0019daf
I20260812 06:53:31.812619 6668 ts_tablet_manager.cc:1939] T 8347033349124342ba23e649d0019daf P 12f64f0540384e9eb9d79f3471132926: Deleting consensus metadata
I20260812 06:53:31.812911 6671 raft_consensus.cc:2258] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:53:31.814025 6670 tablet_replica.cc:330] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: stopping tablet replica
I20260812 06:53:31.814327 5709 catalog_manager.cc:4841] TS 12f64f0540384e9eb9d79f3471132926 (127.17.69.66:45431): tablet 8347033349124342ba23e649d0019daf (table test-table [id=a91990b76b5145e594a519372fb7b24b]) successfully deleted
I20260812 06:53:31.814782 6670 raft_consensus.cc:2229] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:53:31.815210 6670 raft_consensus.cc:2258] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:53:31.816882 6671 ts_tablet_manager.cc:1905] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 06:53:31 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:53:31.823696 6670 ts_tablet_manager.cc:1905] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:53:31.824283 6671 ts_tablet_manager.cc:1918] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:53:31.824831 6671 log.cc:1196] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/8347033349124342ba23e649d0019daf
I20260812 06:53:31.827361 6671 ts_tablet_manager.cc:1939] T 8347033349124342ba23e649d0019daf P 37cc9544485a4289a581fff0c8cbc6fe: Deleting consensus metadata
I20260812 06:53:31.829588 5713 catalog_manager.cc:4841] TS 37cc9544485a4289a581fff0c8cbc6fe (127.17.69.65:44825): tablet 8347033349124342ba23e649d0019daf (table test-table [id=a91990b76b5145e594a519372fb7b24b]) successfully deleted
I20260812 06:53:31.830575 6670 ts_tablet_manager.cc:1918] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:53:31.830745 6670 log.cc:1196] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/8347033349124342ba23e649d0019daf
I20260812 06:53:31.831179 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.803086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.17.69.65:44825)
0812 06:53:31.803259 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.803265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.807733 (+ 4468us) server_negotiation.cc:407] Connection header received
0812 06:53:31.807889 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.807895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.807977 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.808078 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:31.812345 (+ 4267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.813529 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.815843 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.816493 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.816624 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.821947 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:31.821962 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:31.821982 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:31.822031 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:31.826689 (+ 4658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.827704 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.827714 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.827717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.827883 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.828374 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.828380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.828383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.828863 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:31.830173 (+ 1310us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.830474 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.830798 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:53:31.831202 6670 ts_tablet_manager.cc:1939] T 8347033349124342ba23e649d0019daf P 58094ed565d04a6bae4176be6d3dd9da: Deleting consensus metadata
I20260812 06:53:31.831980 6085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:53:31.832687 5709 catalog_manager.cc:4841] TS 58094ed565d04a6bae4176be6d3dd9da (127.17.69.67:41787): tablet 8347033349124342ba23e649d0019daf (table test-table [id=a91990b76b5145e594a519372fb7b24b]) successfully deleted
I20260812 06:53:31.949131 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 5872
I20260812 06:53:31.974403 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 6109
Aug 12 06:53:31 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:53:31.989850 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:31.975193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.17.69.66:45431)
0812 06:53:31.975366 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:31.975371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:31.975499 (+ 128us) server_negotiation.cc:407] Connection header received
0812 06:53:31.975714 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:31.975720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:31.975804 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:31.975888 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:31.977118 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.977947 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.979316 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:31.979749 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:31.979858 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:31.984050 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:31.984062 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:31.984077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:31.984116 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:31.987150 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.987818 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.987825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.987828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.987987 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:31.988330 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:31.988334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:31.988336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:31.988685 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:31.988997 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:31.989272 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 06:53:31.989471 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:53:31.990506 6328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:53:32.117879 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 6109
I20260812 06:53:32.140897 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 6353
Aug 12 06:53:32 dist-test-slave-6zbq krb5kdc[4840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:53:32.154755 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:53:32.141628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.17.69.67:41787)
0812 06:53:32.141813 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 06:53:32.141818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:53:32.141837 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:53:32.141971 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:53:32.141975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:53:32.142048 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:53:32.142128 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:53:32.143336 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:32.144033 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:32.145326 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:53:32.145724 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:53:32.145827 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:53:32.149731 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:53:32.149744 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:53:32.149760 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:53:32.149792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:53:32.152420 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:32.153015 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:32.153021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:32.153023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:32.153141 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:53:32.153424 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:53:32.153427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:53:32.153429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:53:32.153762 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:53:32.153976 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 06:53:32.154213 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 06:53:32.154430 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:53:32.155388 6579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:53:32.240465 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 6353
I20260812 06:53:32.261637 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 5689
I20260812 06:53:32.262821 5783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:53:32.371523 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 5689
I20260812 06:53:32.397308 17685 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:53:32.741865 17685 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:53:32.742166 17685 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:53:33.086149 17685 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:53:33.086 UTC [4911] LOG: received smart shutdown request
2026-08-12 06:53:33.089 UTC [4911] LOG: background worker "logical replication launcher" (PID 4923) exited with exit code 1
2026-08-12 06:53:33.090 UTC [4918] LOG: shutting down
2026-08-12 06:53:33.214 UTC [4911] LOG: database system is shut down
2026-08-12T06:53:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47294 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:53:33 dist-test-slave-6zbq krb5kdc[6699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:53:33 dist-test-slave-6zbq krb5kdc[6699](info): set up 2 sockets
Aug 12 06:53:33 dist-test-slave-6zbq krb5kdc[6699](info): commencing operation
krb5kdc: starting...
W20260812 06:53:34.552311 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:53:34 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:53:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:53:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangeradmin_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangeradmin_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangerlookup_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangerlookup_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
I20260812 06:53:34.862219 17685 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknIa6XT/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/postgres -l logfile start
2026-08-12 06:53:36.915 UTC [6744] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 06:53:36.915 UTC [6744] LOG: listening on IPv4 address "127.17.69.84", port 60069
2026-08-12 06:53:36.918 UTC [6744] LOG: listening on Unix socket "/tmp/.s.PGSQL.60069"
2026-08-12 06:53:36.938 UTC [6750] LOG: database system was shut down at 2026-08-12 06:53:35 UTC
2026-08-12 06:53:36.941 UTC [6744] LOG: database system is ready to accept connections
I20260812 06:53:38.060353 17685 mini_postgres.cc:96] Postgres bound to 60069
2026-08-12 06:53:38.082 UTC [6764] FATAL: database "slave" does not exist
127.17.69.84:60069 - accepting connections
I20260812 06:53:38.083060 17685 mini_ranger.cc:161] Starting Ranger...
I20260812 06:53:38.110378 17685 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 06:53:38.460683 17685 mini_ranger.cc:87] Created ranger Postgres database
I20260812 06:53:38.460784 17685 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin
2026-08-12 06:53:38,727 [I] DB FLAVOR :POSTGRES
2026-08-12 06:53:38,727 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:53:38,727 [I] Checking connection..
2026-08-12 06:53:38,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 06:53:38,977 [I] Checking connection passed.
2026-08-12 06:53:38,978 [I] --------- Verifying version history table ---------
2026-08-12 06:53:38,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:53:39,219 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:53:39,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:53:39,465 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 06:53:39,465 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 06:53:39,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 06:53:39.692 UTC [6875] WARNING: there is no transaction in progress
2026-08-12 06:53:39,700 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 06:53:39,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 06:53:39,952 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 06:53:39,952 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:53:39,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 06:53:40,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 06:53:40,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-12 06:53:40,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:53:40,909 [I] Table x_portal_user does not exist in database ranger
2026-08-12 06:53:40,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:53:41,148 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 06:53:41,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:53:41,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:53:41,633 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 06:53:41,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 06:53:42.499 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.513 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.527 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.540 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.553 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.569 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.627 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.632 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.644 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.653 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.662 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.673 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.682 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.687 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.692 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:42.698 UTC [7055] WARNING: there is no transaction in progress
2026-08-12 06:53:43,198 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 06:53:43,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 06:53:43,438 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 06:53:43,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 06:53:43,672 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 06:53:43,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 06:53:43,925 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 06:53:43,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 06:53:44,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 06:53:44,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-12 06:53:44,653 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 06:53:44,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 06:53:44,902 [I] DB_PATCHES have already been applied
I20260812 06:53:44.908051 17685 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknIa6XT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 06:53:44.908131 17685 mini_ranger.cc:193] Using host: 127.17.69.84
I20260812 06:53:44.927467 17685 mini_ranger.cc:227] Ranger admin URL: http://127.17.69.84:43487
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 6:53:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 06:53:46.229159 17685 mini_ranger.cc:160] Time spent starting Ranger: real 8.146s user 0.002s sys 0.050s
Aug 12, 2026 6:54:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 6:54:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 6:54:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 6:54:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 6:54:05 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 06:54:05.875334 17685 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.69.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.69.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangerkms_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.69.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/rangerkms_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.69.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.69.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.69.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/HTTP_127.17.69.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 06:54:06 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517646, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 06:54:06.038141 17685 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 06:54:06.059135 17685 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 06:54:06.707058 17685 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 06:54:06.707173 17685 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms
I20260812 06:54:06.707219 17685 mini_ranger_kms.cc:227] Using postgres at 127.17.69.84:60069
2026-08-12 06:54:07,036 [I] DB FLAVOR :POSTGRES
2026-08-12 06:54:07,037 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 06:54:07,037 [I] Checking connection
2026-08-12 06:54:07,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 06:54:07,276 [I] connection success
2026-08-12 06:54:07,276 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 06:54:07,276 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 06:54:07,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 06:54:07,504 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 06:54:07,504 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 06:54:07,504 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 06:54:07,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.69.84:60069/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 06:54:07,749 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 06:54:08.198172 17685 mini_ranger_kms.cc:310] Created kms service
I20260812 06:54:08.349984 17685 mini_ranger_kms.cc:326] Created kudu user
I20260812 06:54:08.389554 17685 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 06:54:08 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517646, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:08.753440 17685 mini_ranger_kms.cc:384] Added ranger policy
I20260812 06:54:08.753764 17685 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknIa6XT/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknIa6XT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 06:54:08.753808 17685 mini_ranger_kms.cc:240] Using host: 127.17.69.84
I20260812 06:54:08.774623 17685 mini_ranger_kms.cc:276] Ranger KMS PID: 7367
I20260812 06:54:08.774731 17685 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.69.84:52171
Aug 12, 2026 6:54:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 6:54:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 6:54:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 6:54:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 6:54:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 6:54:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 06:54:10 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:11.443988 17685 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.406s user 0.052s sys 0.304s
I20260812 06:54:11.444263 17685 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 06:54:11.444430 17685 mini_ranger_kms.cc:401] 127.17.69.84:52171/kms/v1/keys
Aug 12 06:54:11 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
Aug 12 06:54:11 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.69.84@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
Aug 12 06:54:14 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:54:14.316021 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45195
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45195
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:14.639516 7458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:14.639777 7458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:14.639889 7458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:14.646850 7458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:14.646948 7458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:14.647002 7458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:14.647075 7458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:14.647137 7458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:14.656795 7458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.69.84:52171/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45195
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:14.657984 7458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:14.659354 7458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:14.671384 7467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:14.671613 7468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:14.671831 7472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:14.672192 7458 server_base.cc:1034] running on GCE node
I20260812 06:54:14.672858 7458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:14.674160 7458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:14.675405 7458 hybrid_clock.cc:648] HybridClock initialized: now 1786517654675329 us; error 43 us; skew 500 ppm
I20260812 06:54:14.675779 7458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:14 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:14.679843 7458 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:14.681496 7458 webserver.cc:460] Webserver started at http://127.17.69.126:38061/ using document root <none> and password file <none>
I20260812 06:54:14.682334 7458 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:14.682490 7458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:14.682817 7458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:54:14 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:14.740783 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "1d852bd9053445b598a0c2456ad97d58"
format_stamp: "Formatted at 2026-08-12 06:54:14 on dist-test-slave-6zbq"
server_key: "a685c0f7702b36c3dc48a99a5109f25f"
server_key_iv: "dfa8d92c7ab675cf4234cf800c3103f5"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:14.741902 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "1d852bd9053445b598a0c2456ad97d58"
format_stamp: "Formatted at 2026-08-12 06:54:14 on dist-test-slave-6zbq"
server_key: "a685c0f7702b36c3dc48a99a5109f25f"
server_key_iv: "dfa8d92c7ab675cf4234cf800c3103f5"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:14.748510 7458 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260812 06:54:14.767984 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:14.769583 7458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:14.769799 7458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "1d852bd9053445b598a0c2456ad97d58"
format_stamp: "Formatted at 2026-08-12 06:54:14 on dist-test-slave-6zbq"
server_key: "a685c0f7702b36c3dc48a99a5109f25f"
server_key_iv: "dfa8d92c7ab675cf4234cf800c3103f5"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:14.770061 7458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:14.788630 7458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:14.793063 7458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:14.793375 7458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:14.810968 7458 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:45195
I20260812 06:54:14.811007 7573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:45195 every 8 connection(s)
I20260812 06:54:14.812175 7458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:14.815613 7574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:14.820803 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 7458
I20260812 06:54:14.821007 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:54:14.821844 17685 external_mini_cluster.cc:1428] Setting key 8cafeadd5a011ce9f66283b07b23d875
I20260812 06:54:14.826422 7574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Bootstrap starting.
I20260812 06:54:14.829212 7574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:14.830154 7574 log.cc:824] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:14.833158 7574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: No bootstrap required, opened a new log
Aug 12 06:54:14 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:14.842618 7574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:14.843051 7574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:14.843210 7574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d852bd9053445b598a0c2456ad97d58, State: Initialized, Role: FOLLOWER
I20260812 06:54:14.844005 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:14.823963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.17.69.126:45195)
0812 06:54:14.824727 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:14.824739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:14.824778 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:54:14.826330 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:14.826382 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:14.826756 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:14.827079 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:14.828421 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:14.829707 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:14.831187 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:14.832102 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:14.832298 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:14.836086 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:14.836097 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:14.836114 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:14.836165 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:14.839927 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:14.840516 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:14.840525 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:14.840533 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:14.840730 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:14.841042 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:14.841047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:14.841050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:14.841752 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:14.842028 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:14.842363 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:54:14.842736 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":362,"threads_started":1}
I20260812 06:54:14.843856 7574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:14.844156 7574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:14.844331 7574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:14.844499 7574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:14.846721 7574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:14.847167 7574 leader_election.cc:302] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d852bd9053445b598a0c2456ad97d58; no voters:
I20260812 06:54:14.847694 7574 leader_election.cc:288] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:14.847813 7583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:14.848469 7583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 LEADER]: Becoming Leader. State: Replica: 1d852bd9053445b598a0c2456ad97d58, State: Running, Role: LEADER
I20260812 06:54:14.848987 7583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:14.849709 7574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:14.851660 7586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d852bd9053445b598a0c2456ad97d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } } }
I20260812 06:54:14.851660 7587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d852bd9053445b598a0c2456ad97d58. Latest consensus state: current_term: 1 leader_uuid: "1d852bd9053445b598a0c2456ad97d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } } }
I20260812 06:54:14.852077 7587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:14.852075 7586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:14.853829 7591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:14.858580 7591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:14.861536 7574 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 06:54:14.876773 7591 catalog_manager.cc:1347] Generated new cluster ID: 33bf50b20faf489d99899c9201ede997
I20260812 06:54:14.876905 7591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:14.892160 7591 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:14.893898 7591 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:14.905172 7591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Generated new TSK 0
I20260812 06:54:14.906097 7591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:54:15.587692 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:15 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:54:15.952454 7638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:15.952683 7638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:15.952795 7638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:15.959858 7638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:15.960012 7638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:15.960206 7638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:15.970888 7638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:15.972024 7638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:15.973237 7638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:15.987250 7660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:15.996261 7661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:15.996681 7669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:15.997103 7638 server_base.cc:1034] running on GCE node
I20260812 06:54:15.997658 7638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:15.998581 7638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:15.999846 7638 hybrid_clock.cc:648] HybridClock initialized: now 1786517655999772 us; error 70 us; skew 500 ppm
I20260812 06:54:16.000129 7638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:16.003517 7638 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:16.010802 7638 webserver.cc:460] Webserver started at http://127.17.69.65:43047/ using document root <none> and password file <none>
I20260812 06:54:16.011447 7638 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:16.011560 7638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:16.011862 7638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:16.029425 7638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "56e47390691949df92b5ca1ace5ea8d6"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "bcbf679ffdee6cb69e47678be73697b3"
server_key_iv: "0676986e8feaf9854255b44feb3f1a15"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.030680 7638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "56e47390691949df92b5ca1ace5ea8d6"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "bcbf679ffdee6cb69e47678be73697b3"
server_key_iv: "0676986e8feaf9854255b44feb3f1a15"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.048673 7638 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.016s sys 0.001s
I20260812 06:54:16.068717 7686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:16.070514 7638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:54:16.070737 7638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "56e47390691949df92b5ca1ace5ea8d6"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "bcbf679ffdee6cb69e47678be73697b3"
server_key_iv: "0676986e8feaf9854255b44feb3f1a15"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.071070 7638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:16.097453 7638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:16.102448 7638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:16.102831 7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:16.103981 7638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:16.105732 7638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:16.105834 7638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:16.106070 7638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:16.106148 7638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:16.143222 7638 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:44089
I20260812 06:54:16.143276 7872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:44089 every 8 connection(s)
I20260812 06:54:16.144876 7638 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:16.146920 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 7638
I20260812 06:54:16.147131 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:16.148020 17685 external_mini_cluster.cc:1428] Setting key 96954db5d7c4469cb46d4da1cd1cbd99
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
I20260812 06:54:16.191947 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:16.152153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:45919 (local address 127.17.69.126:45195)
0812 06:54:16.170731 (+ 18578us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:16.170737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:16.174162 (+ 3425us) server_negotiation.cc:407] Connection header received
0812 06:54:16.174229 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:16.174234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:16.174333 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:16.174461 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:16.176941 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:16.178087 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:16.180290 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:16.180898 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:16.181016 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:16.184910 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:16.184920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:16.184925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:16.184969 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:16.188337 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:16.189269 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:16.189275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:16.189278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:16.189435 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:16.189938 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:16.189943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:16.189947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:16.190330 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:16.190509 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:16.191371 (+ 862us) server_negotiation.cc:299] Negotiation successful
0812 06:54:16.191589 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18444,"thread_start_us":18332,"threads_started":1}
I20260812 06:54:16.192914 7874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:16.151509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.65:45919)
0812 06:54:16.171298 (+ 19789us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:16.171346 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:16.173823 (+ 2477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:16.175402 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:16.175414 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:16.175972 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:16.176737 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:16.176761 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:16.178240 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:16.178246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:16.180086 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:16.180119 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:16.181153 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:16.181162 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:16.181437 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:16.182137 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:16.182162 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:16.184588 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:16.188506 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:16.188516 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:16.188531 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:16.189115 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:16.189553 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:16.189558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:16.189562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:16.189808 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:16.190497 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:16.190505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:16.190916 (+ 411us) client_negotiation.cc:769] Sending connection context
0812 06:54:16.191219 (+ 303us) client_negotiation.cc:240] Negotiation successful
0812 06:54:16.191713 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19482,"thread_start_us":19253,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
I20260812 06:54:16.194535 7873 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:16.194839 7873 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:16.195484 7873 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:16.197460 7509 ts_manager.cc:194] Registered new tserver with Master: 56e47390691949df92b5ca1ace5ea8d6 (127.17.69.65:44089)
I20260812 06:54:16.198910 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:45919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:54:16.232163 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
W20260812 06:54:16.623399 7881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:16.623648 7881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:16.623812 7881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:16.630846 7881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:16.630949 7881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:16.631103 7881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:16.643093 7881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:16.644312 7881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:16.645995 7881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:16.674526 7895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:16.675534 7894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:16.716609 7881 server_base.cc:1034] running on GCE node
W20260812 06:54:16.718850 7901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:16.720476 7881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:16.738687 7881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:16.740048 7881 hybrid_clock.cc:648] HybridClock initialized: now 1786517656739936 us; error 104 us; skew 500 ppm
I20260812 06:54:16.740406 7881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:16.744854 7881 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:16.746996 7881 webserver.cc:460] Webserver started at http://127.17.69.66:34361/ using document root <none> and password file <none>
I20260812 06:54:16.747879 7881 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:16.748015 7881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:16.748380 7881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:54:16 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:16.767441 7881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "0bcc2aa31fc918c0c961265c02120205"
server_key_iv: "f6714dc2455c95a52b753492d166a5e2"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.768472 7881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "0bcc2aa31fc918c0c961265c02120205"
server_key_iv: "f6714dc2455c95a52b753492d166a5e2"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.777555 7881 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260812 06:54:16.851426 7915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:16.860224 7881 fs_manager.cc:725] Time spent opening block manager: real 0.027s user 0.005s sys 0.005s
I20260812 06:54:16.860450 7881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "0bcc2aa31fc918c0c961265c02120205"
server_key_iv: "f6714dc2455c95a52b753492d166a5e2"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:16.860733 7881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:16.922187 7881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:16.932035 7881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:16.932437 7881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:16.946751 7881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:16.959818 7881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:16.959993 7881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:16.960188 7881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:16.960261 7881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:17.131340 7881 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:35301
I20260812 06:54:17.131358 8121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:35301 every 8 connection(s)
I20260812 06:54:17.132614 7881 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:17.136988 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 7881
I20260812 06:54:17.137171 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:17.137733 17685 external_mini_cluster.cc:1428] Setting key 21e6008935e332eae34b0c762838282f
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:17.170769 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.138865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:52467 (local address 127.17.69.126:45195)
0812 06:54:17.139951 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:17.139957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:17.151611 (+ 11654us) server_negotiation.cc:407] Connection header received
0812 06:54:17.153931 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:17.153937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:17.154019 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:17.154091 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:17.156084 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.157026 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.159271 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.159866 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.159963 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.164060 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:17.164066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:17.164071 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:17.164109 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:17.167331 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.168142 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.168148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.168152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.168325 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.168845 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.168850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.168853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.169161 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:17.169296 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:17.170184 (+ 888us) server_negotiation.cc:299] Negotiation successful
0812 06:54:17.170434 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":834,"threads_started":1}
I20260812 06:54:17.173029 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.136133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.66:52467)
0812 06:54:17.151470 (+ 15337us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:17.151513 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:17.153590 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:17.154612 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:17.154621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:17.155161 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:17.155893 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:17.155916 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.157221 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.157225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:17.159106 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:17.159130 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.160249 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.160255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:17.160525 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.161254 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:17.161281 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:17.163843 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:17.167495 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:17.167502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:17.167516 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:17.168001 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:17.168466 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:17.168470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:17.168474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:17.168724 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:17.169386 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:17.169393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:17.169733 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 06:54:17.171273 (+ 1540us) client_negotiation.cc:240] Negotiation successful
0812 06:54:17.171743 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15050,"thread_start_us":21381,"threads_started":1}
I20260812 06:54:17.174793 8122 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:17.175101 8122 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:17.175935 8122 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:17.177641 7509 ts_manager.cc:194] Registered new tserver with Master: b897aa7c5abf45d2b8c859b9eaae7c3c (127.17.69.66:35301)
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
I20260812 06:54:17.178630 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:52467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
I20260812 06:54:17.201565 7873 heartbeater.cc:503] Master 127.17.69.126:45195 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:54:17.208832 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:17.527697 8132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:17.527963 8132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:17.528088 8132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:17.535126 8132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:17.535236 8132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:17.535396 8132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:17.545198 8132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:17.546408 8132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:17.547629 8132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:17.566799 8139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:17.567781 8140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:17.570183 8144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:17.570636 8132 server_base.cc:1034] running on GCE node
I20260812 06:54:17.571213 8132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:17.572151 8132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:17.573362 8132 hybrid_clock.cc:648] HybridClock initialized: now 1786517657573314 us; error 40 us; skew 500 ppm
I20260812 06:54:17.573659 8132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:17.580442 8132 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:17.583568 8132 webserver.cc:460] Webserver started at http://127.17.69.67:33633/ using document root <none> and password file <none>
I20260812 06:54:17.584208 8132 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:17.584332 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:17.584630 8132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:17.600812 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "ac0f6eaa01574982903fec84749e7c5e"
format_stamp: "Formatted at 2026-08-12 06:54:17 on dist-test-slave-6zbq"
server_key: "dec8eb90a7f186d3b5e129904a043dca"
server_key_iv: "c092f27081601b6eed72b1d1b6dee6cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:17.601601 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "ac0f6eaa01574982903fec84749e7c5e"
format_stamp: "Formatted at 2026-08-12 06:54:17 on dist-test-slave-6zbq"
server_key: "dec8eb90a7f186d3b5e129904a043dca"
server_key_iv: "c092f27081601b6eed72b1d1b6dee6cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:17.607777 8132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 06:54:17.642696 8156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:17.644184 8132 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.016s sys 0.004s
I20260812 06:54:17.644358 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "ac0f6eaa01574982903fec84749e7c5e"
format_stamp: "Formatted at 2026-08-12 06:54:17 on dist-test-slave-6zbq"
server_key: "dec8eb90a7f186d3b5e129904a043dca"
server_key_iv: "c092f27081601b6eed72b1d1b6dee6cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:17.644572 8132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:17.661936 8132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:17.665995 8132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:17.666265 8132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:17.667388 8132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:17.668915 8132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:17.668992 8132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:17.669113 8132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:17.669173 8132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:17.722980 8132 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:45607
I20260812 06:54:17.723001 8347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:45607 every 8 connection(s)
I20260812 06:54:17.724283 8132 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:17.732717 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 8132
I20260812 06:54:17.732903 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:17.733471 17685 external_mini_cluster.cc:1428] Setting key f4e2c1ba8ddbacf99fcb03ba602e17e0
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:17.756006 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.727879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.67:40563)
0812 06:54:17.739317 (+ 11438us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:17.739372 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:17.741019 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:17.741721 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:17.741729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:17.742146 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:17.742761 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:17.742778 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.743802 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.743806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:17.744999 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:17.745021 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.745829 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.745837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:17.746067 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.746693 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:17.746715 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:17.749593 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:17.752514 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:17.752522 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:17.752537 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:17.752941 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:17.753284 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:17.753289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:17.753293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:17.753476 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:17.753974 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:17.753981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:17.754444 (+ 463us) client_negotiation.cc:769] Sending connection context
0812 06:54:17.754722 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 06:54:17.755106 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11163,"thread_start_us":10949,"threads_started":1}
I20260812 06:54:17.756973 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.728580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:40563 (local address 127.17.69.126:45195)
0812 06:54:17.738774 (+ 10194us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:17.738781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:17.739775 (+ 994us) server_negotiation.cc:407] Connection header received
0812 06:54:17.741283 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:17.741296 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:17.741389 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:17.741480 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:17.742928 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.743657 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.745144 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.745568 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.745745 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.749830 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:17.749835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:17.749838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:17.749869 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:17.752389 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.753059 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.753065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.753067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.753187 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.753592 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.753595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.753597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.753855 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:17.753983 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:17.756449 (+ 2466us) server_negotiation.cc:299] Negotiation successful
0812 06:54:17.756663 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10060,"thread_start_us":9948,"threads_started":1}
I20260812 06:54:17.757961 8349 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:17.758247 8349 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:17.758965 8349 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:17.760416 7509 ts_manager.cc:194] Registered new tserver with Master: ac0f6eaa01574982903fec84749e7c5e (127.17.69.67:45607)
I20260812 06:54:17.761217 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:40563
I20260812 06:54:17.762609 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:17.764645 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 7638
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:54:17.785610 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.765758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.17.69.65:44089)
0812 06:54:17.770803 (+ 5045us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:17.770809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:17.770832 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:54:17.770886 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:17.770891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:17.771057 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:17.771196 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:17.772618 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.773579 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.775179 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.775794 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.775908 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.780162 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:17.780173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:17.780184 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:17.780215 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:17.782832 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.783522 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.783530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.783533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.783694 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.784044 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.784049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.784053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.784361 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:17.784649 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:17.784928 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 06:54:17.785162 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4900,"thread_start_us":4764,"threads_started":1}
I20260812 06:54:17.786737 7868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:17.874271 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 7638
I20260812 06:54:17.900269 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 7881
Aug 12 06:54:17 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:54:17.915171 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:17.901108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37736 (local address 127.17.69.66:35301)
0812 06:54:17.901636 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:17.901643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:17.901667 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:54:17.901723 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:17.901729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:17.901943 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:17.902078 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:17.903294 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.904296 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.905571 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:17.906104 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:17.906216 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:17.910114 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:17.910125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:17.910137 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:17.910170 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:17.912825 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.913384 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.913389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.913391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.913517 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:17.913891 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:17.913895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:17.913897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:17.914221 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:17.914493 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:17.914707 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 06:54:17.914903 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":255,"threads_started":1}
I20260812 06:54:17.916316 8117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:17.994951 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 7881
I20260812 06:54:18.020296 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 8132
Aug 12 06:54:18 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:54:18.035023 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:18.021243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.17.69.67:45607)
0812 06:54:18.021829 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:18.021836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:18.021864 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:54:18.021921 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:18.021927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:18.022129 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:18.022257 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:18.023507 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:18.024498 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:18.025621 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:18.026128 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:18.026233 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:18.030224 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:18.030234 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:18.030247 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:18.030282 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:18.032788 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:18.033404 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:18.033409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:18.033411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:18.033526 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:18.033797 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:18.033800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:18.033802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:18.034097 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:18.034334 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:18.034558 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 06:54:18.034743 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":292,"threads_started":1}
I20260812 06:54:18.036193 8341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:18.115540 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 8132
I20260812 06:54:18.135936 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 7458
I20260812 06:54:18.137089 7558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:18.238827 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 7458
I20260812 06:54:18.267788 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45195
--webserver_interface=127.17.69.126
--webserver_port=38061
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45195
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:18.590763 8365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:18.591035 8365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:18.591189 8365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:18.598227 8365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:18.598331 8365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:18.598428 8365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:18.598508 8365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:18.598572 8365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:18.610740 8365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45195
--ranger_config_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=38061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:18.611878 8365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:18.613077 8365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:18.624804 8379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:18.625147 8375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:18.625337 8376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:18.625955 8365 server_base.cc:1034] running on GCE node
I20260812 06:54:18.626876 8365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:18.628571 8365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:18.629854 8365 hybrid_clock.cc:648] HybridClock initialized: now 1786517658629791 us; error 42 us; skew 500 ppm
I20260812 06:54:18.630265 8365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:18 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517658, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:18.633945 8365 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:18.635356 8365 webserver.cc:460] Webserver started at http://127.17.69.126:38061/ using document root <none> and password file <none>
I20260812 06:54:18.635931 8365 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:18.636046 8365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:54:18 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517658, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:18.714774 8365 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:54:18.717622 8396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:18.718962 8365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:18.719132 8365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "1d852bd9053445b598a0c2456ad97d58"
format_stamp: "Formatted at 2026-08-12 06:54:14 on dist-test-slave-6zbq"
server_key: "a685c0f7702b36c3dc48a99a5109f25f"
server_key_iv: "dfa8d92c7ab675cf4234cf800c3103f5"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:18.720064 8365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:18.758093 8365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:18.761986 8365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:18.762246 8365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:18.779129 8365 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:45195
I20260812 06:54:18.779163 8483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:45195 every 8 connection(s)
I20260812 06:54:18.780440 8365 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:18.782879 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 8365
I20260812 06:54:18.783903 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:44089
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:45195
--webserver_port=43047
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:18.788594 8484 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:54:18.792712 8484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Bootstrap starting.
I20260812 06:54:18.811446 8484 log.cc:824] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:18.820343 8484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:54:18.820827 8484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Bootstrap complete.
I20260812 06:54:18.829552 8484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:18.830387 8484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d852bd9053445b598a0c2456ad97d58, State: Initialized, Role: FOLLOWER
I20260812 06:54:18.830941 8484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:18.831135 8484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:18.831252 8484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:18.831413 8484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:54:18.834954 8484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:18.835394 8484 leader_election.cc:302] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d852bd9053445b598a0c2456ad97d58; no voters:
I20260812 06:54:18.835955 8484 leader_election.cc:288] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:54:18.836090 8493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:54:18.837095 8493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [term 2 LEADER]: Becoming Leader. State: Replica: 1d852bd9053445b598a0c2456ad97d58, State: Running, Role: LEADER
I20260812 06:54:18.837677 8493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } }
I20260812 06:54:18.837927 8484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:18.840037 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d852bd9053445b598a0c2456ad97d58. Latest consensus state: current_term: 2 leader_uuid: "1d852bd9053445b598a0c2456ad97d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } } }
I20260812 06:54:18.840409 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:18.840898 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d852bd9053445b598a0c2456ad97d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852bd9053445b598a0c2456ad97d58" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45195 } } }
I20260812 06:54:18.841207 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:18.851224 8504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:18.857059 8504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:18.858474 8504 catalog_manager.cc:1259] Loaded cluster ID: 33bf50b20faf489d99899c9201ede997
I20260812 06:54:18.858614 8504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:18.862457 8504 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:18.863863 8504 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Loaded TSK: 0
I20260812 06:54:18.865337 8504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:54:18.870042 8484 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260812 06:54:19.184093 8488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:19.184487 8488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:19.184666 8488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:19.195878 8488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:19.196048 8488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:19.196254 8488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:19.211205 8488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=43047
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:19.212882 8488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:19.215459 8488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:19.266393 8555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:19.275111 8556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:19.285274 8488 server_base.cc:1034] running on GCE node
W20260812 06:54:19.285543 8559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:19.286368 8488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:19.287739 8488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:19.289005 8488 hybrid_clock.cc:648] HybridClock initialized: now 1786517659288863 us; error 128 us; skew 500 ppm
I20260812 06:54:19.289333 8488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:19 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:19.293941 8488 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:19.314015 8488 webserver.cc:460] Webserver started at http://127.17.69.65:43047/ using document root <none> and password file <none>
I20260812 06:54:19.315128 8488 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:19.315357 8488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:54:19 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:19.375496 8488 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.001s sys 0.008s
I20260812 06:54:19.382305 8571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:19.384148 8488 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.001s
I20260812 06:54:19.384346 8488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "56e47390691949df92b5ca1ace5ea8d6"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "bcbf679ffdee6cb69e47678be73697b3"
server_key_iv: "0676986e8feaf9854255b44feb3f1a15"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:19.385785 8488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:19.424043 8488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:19.428974 8488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:19.429302 8488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:19.430670 8488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:19.432453 8488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:19.432576 8488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:19.432685 8488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:19.432763 8488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:19.532444 8488 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:44089
I20260812 06:54:19.532503 8756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:44089 every 8 connection(s)
I20260812 06:54:19.534034 8488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:19.543111 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 8488
I20260812 06:54:19.543905 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:35301
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:45195
--webserver_port=34361
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:19 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:19.567587 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:19.537254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:44681 (local address 127.17.69.126:45195)
0812 06:54:19.546846 (+ 9592us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:19.546863 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:19.546915 (+ 52us) server_negotiation.cc:407] Connection header received
0812 06:54:19.547265 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:19.547278 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:19.547609 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:19.547967 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:19.550256 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:19.552053 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:19.554294 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:19.554938 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:19.555110 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:19.559004 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:19.559016 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:19.559033 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:19.559077 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:19.562730 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:19.563549 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:19.563557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:19.563565 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:19.563756 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:19.564236 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:19.564241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:19.564245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:19.564814 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:19.564979 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:19.565916 (+ 937us) server_negotiation.cc:299] Negotiation successful
0812 06:54:19.566315 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9333,"thread_start_us":9136,"threads_started":1}
I20260812 06:54:19.569113 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:19.537258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.65:44681)
0812 06:54:19.538862 (+ 1604us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:19.538917 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:19.546046 (+ 7129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:19.548706 (+ 2660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:19.548716 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:19.549306 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:19.550004 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:19.550031 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:19.552255 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:19.552259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:19.554158 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:19.554171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:19.555174 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:19.555182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:19.555438 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:19.556197 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:19.556225 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:19.558714 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:19.562895 (+ 4181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:19.562904 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:19.562918 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:19.563425 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:19.563887 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:19.563891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:19.563895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:19.564128 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:19.565059 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:19.565067 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:19.565415 (+ 348us) client_negotiation.cc:769] Sending connection context
0812 06:54:19.567432 (+ 2017us) client_negotiation.cc:240] Negotiation successful
0812 06:54:19.567951 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"thread_start_us":1082,"threads_started":1}
I20260812 06:54:19.572227 8758 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:19.572532 8758 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:19.573347 8758 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
W20260812 06:54:19.576565 8758 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:45195 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:54:19.576792 8758 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:19.577062 8758 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:19.578310 8758 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:19.578579 8758 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
W20260812 06:54:19.579722 8758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:54:19.921924 8764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:19.922173 8764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:19.922298 8764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:19.929656 8764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:19.929746 8764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:19.929878 8764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:19.939329 8764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:35301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=34361
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:19.940552 8764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:19.941859 8764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:19.955597 8787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:19.955627 8788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:19.956112 8790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:19.956969 8764 server_base.cc:1034] running on GCE node
I20260812 06:54:19.957532 8764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:19.958460 8764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:19.959703 8764 hybrid_clock.cc:648] HybridClock initialized: now 1786517659959646 us; error 37 us; skew 500 ppm
I20260812 06:54:19.960019 8764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:19 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:19.963325 8764 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:19.967650 8764 webserver.cc:460] Webserver started at http://127.17.69.66:34361/ using document root <none> and password file <none>
I20260812 06:54:19.968462 8764 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:19.968626 8764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:54:19 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:19.989341 8764 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:54:19.991936 8805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:19.993184 8764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:54:19.993343 8764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c"
format_stamp: "Formatted at 2026-08-12 06:54:16 on dist-test-slave-6zbq"
server_key: "0bcc2aa31fc918c0c961265c02120205"
server_key_iv: "f6714dc2455c95a52b753492d166a5e2"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:19.994256 8764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:20.008630 8764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:20.012425 8764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:20.012658 8764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:20.013585 8764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:20.014904 8764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:20.015004 8764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:20.015108 8764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:20.015178 8764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:20.081558 8764 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:35301
I20260812 06:54:20.081601 8994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:35301 every 8 connection(s)
I20260812 06:54:20.083297 8764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:20.086774 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 8764
I20260812 06:54:20.087572 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.69.84:52171/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:45607
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:45195
--webserver_port=33633
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:20.122864 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.085947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58953 (local address 127.17.69.126:45195)
0812 06:54:20.087382 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.087389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.100147 (+ 12758us) server_negotiation.cc:407] Connection header received
0812 06:54:20.103510 (+ 3363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.103516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.103606 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.103721 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:20.106899 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.108043 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.110434 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.111075 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.111173 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.115553 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:20.115561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:20.115566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:20.115606 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:20.119061 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.119994 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.119999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.120002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.120157 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.120693 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.120698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.120702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.121057 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:20.121220 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.122174 (+ 954us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.122427 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1311,"thread_start_us":1205,"threads_started":1}
I20260812 06:54:20.125208 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.087916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.66:58953)
0812 06:54:20.099959 (+ 12043us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:20.100008 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:20.103194 (+ 3186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:20.105219 (+ 2025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:20.105231 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:20.105820 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:20.106672 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.106698 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.108265 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.108270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.110172 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.110201 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.111496 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.111504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.111779 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.112545 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:20.112576 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:20.115308 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:20.119238 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:20.119246 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:20.119262 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:20.119850 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:20.120311 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:20.120315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:20.120319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:20.120558 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:20.121320 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:20.121327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:20.121736 (+ 409us) client_negotiation.cc:769] Sending connection context
0812 06:54:20.123168 (+ 1432us) client_negotiation.cc:240] Negotiation successful
0812 06:54:20.123703 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11725,"thread_start_us":11500,"threads_started":1}
I20260812 06:54:20.127075 9004 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:20.127393 9004 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:20.128270 9004 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:20.130209 8428 ts_manager.cc:194] Registered new tserver with Master: b897aa7c5abf45d2b8c859b9eaae7c3c (127.17.69.66:35301)
I20260812 06:54:20.132372 8428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:58953
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 06:54:20.540788 9011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:20.541107 9011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:20.541302 9011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:20.552320 9011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:20.552503 9011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:20.552731 9011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:20.567945 9011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.69.84:52171/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:45607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=33633
--tserver_master_addrs=127.17.69.126:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:20.569540 9011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:20.571298 9011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:54:20.579849 8758 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:20.580245 8758 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:20.581478 8428 ts_manager.cc:194] Registered new tserver with Master: 56e47390691949df92b5ca1ace5ea8d6 (127.17.69.65:44089)
I20260812 06:54:20.582643 8428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:44681
W20260812 06:54:20.588224 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:20.588248 9019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:20.595162 9024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:20.596108 9011 server_base.cc:1034] running on GCE node
I20260812 06:54:20.596807 9011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:20.598805 9011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:20.600047 9011 hybrid_clock.cc:648] HybridClock initialized: now 1786517660599981 us; error 53 us; skew 500 ppm
I20260812 06:54:20.600453 9011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:20.605890 9011 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:20.618757 9011 webserver.cc:460] Webserver started at http://127.17.69.67:33633/ using document root <none> and password file <none>
I20260812 06:54:20.619592 9011 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:20.619727 9011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:20.668992 9011 fs_manager.cc:709] Time spent opening directory manager: real 0.030s user 0.012s sys 0.014s
I20260812 06:54:20.672128 9041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:20.673480 9011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:54:20.673643 9011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "ac0f6eaa01574982903fec84749e7c5e"
format_stamp: "Formatted at 2026-08-12 06:54:17 on dist-test-slave-6zbq"
server_key: "dec8eb90a7f186d3b5e129904a043dca"
server_key_iv: "c092f27081601b6eed72b1d1b6dee6cf"
server_key_version: "kuduclusterkey@0"
I20260812 06:54:20.674754 9011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:20.727855 9011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:20.732971 9011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:20.733279 9011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:20.734489 9011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:20.736263 9011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:20.736377 9011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:20.736490 9011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:20.736574 9011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:20.803566 9011 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:45607
I20260812 06:54:20.803647 9230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:45607 every 8 connection(s)
I20260812 06:54:20.805505 9011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:20.809211 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 9011
Aug 12 06:54:20 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:20.845077 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.822692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.17.69.126:45195)
0812 06:54:20.826457 (+ 3765us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.826462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.826477 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:54:20.826525 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.826529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.826598 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.828688 (+ 2090us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:20.828929 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.830030 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.831976 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.833189 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.837214 (+ 4025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.837741 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:20.837747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:20.837752 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:20.837783 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:20.841371 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.842184 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.842188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.842191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.842314 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.842821 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.842825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.842828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.843131 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:20.844408 (+ 1277us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.844551 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.844747 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3644,"thread_start_us":3463,"threads_started":1}
I20260812 06:54:20.846014 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.808419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:60405 (local address 127.17.69.126:45195)
0812 06:54:20.822786 (+ 14367us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.822793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.824060 (+ 1267us) server_negotiation.cc:407] Connection header received
0812 06:54:20.826053 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.826063 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.826153 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.826271 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:20.828448 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.829519 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.831996 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.832605 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.832711 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.836652 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:20.836658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:20.836662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:20.836698 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:20.840529 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.841405 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.841409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.841413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.841548 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.842073 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.842078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.842081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.842450 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:20.842577 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.845482 (+ 2905us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.845654 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":14220,"spinlock_wait_cycles":4352,"thread_start_us":14068,"threads_started":1}
I20260812 06:54:20.845110 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.809038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45195 (local address 127.17.69.67:60405)
0812 06:54:20.823909 (+ 14871us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:20.823962 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:20.825750 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:20.826704 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:20.826717 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:20.827421 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:20.828243 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.828269 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.829713 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.829719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.831668 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.831685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.832827 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.832835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.833096 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.833854 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:20.833883 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:20.836400 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:20.840683 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:20.840691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:20.840706 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:20.841267 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:20.841673 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:20.841678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:20.841681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:20.841937 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:20.842648 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:20.842655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:20.843049 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 06:54:20.843323 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 06:54:20.843824 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14602,"thread_start_us":14406,"threads_started":1}
I20260812 06:54:20.847807 9231 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45195
I20260812 06:54:20.848114 9231 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:20.848799 9231 heartbeater.cc:511] Master 127.17.69.126:45195 requested a full tablet report, sending...
I20260812 06:54:20.850059 8428 ts_manager.cc:194] Registered new tserver with Master: ac0f6eaa01574982903fec84749e7c5e (127.17.69.67:45607)
I20260812 06:54:20.850853 8428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:60405
I20260812 06:54:20.860309 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:20.876088 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.863528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.17.69.126:45195)
0812 06:54:20.863738 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.863745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.864235 (+ 490us) server_negotiation.cc:407] Connection header received
0812 06:54:20.864399 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.864404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.864469 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.864547 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:20.866040 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.866789 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.868140 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.868580 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.868689 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.871130 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:20.871135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:20.871138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:20.871172 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:20.873550 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.874161 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.874167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.874171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.874291 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:20.874713 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:20.874718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:20.874722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:20.874975 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:20.875229 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.875518 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.875723 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 06:54:20.880893 8428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:20.890118 8428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:20.966423 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.904834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44089 (local address 127.0.0.1:34248)
0812 06:54:20.951642 (+ 46808us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:20.951660 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:20.951773 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:20.955982 (+ 4209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:20.955991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:20.956029 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:20.956442 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.956454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.960082 (+ 3628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.960088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.963170 (+ 3082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.963191 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.965348 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.965354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.965546 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.965601 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 06:54:20.965755 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 06:54:20.965961 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46684,"mutex_wait_us":1,"thread_start_us":49518,"threads_started":1}
I20260812 06:54:20.967475 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.904583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.17.69.65:44089)
0812 06:54:20.954999 (+ 50416us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.955008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.955039 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:54:20.955149 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.955158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.955415 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.955628 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:20.956678 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.959917 (+ 3239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.963351 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.964845 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.964960 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.966903 (+ 1943us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.966976 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.967134 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50240,"thread_start_us":54828,"threads_started":1}
I20260812 06:54:20.970504 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.904179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:35301 (local address 127.0.0.1:37744)
0812 06:54:20.950742 (+ 46563us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:20.950805 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:20.951050 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:20.963329 (+ 12279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:20.963335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:20.963363 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:20.963715 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.963726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.965603 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.965608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.967835 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.967850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.969731 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.969739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.969907 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.969941 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:54:20.970011 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 06:54:20.970135 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46214,"thread_start_us":58310,"threads_started":1}
I20260812 06:54:20.971747 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.904741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37744 (local address 127.17.69.66:35301)
0812 06:54:20.962145 (+ 57404us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.962153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.962196 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:54:20.962295 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.962303 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.962862 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.963068 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:20.963901 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.965438 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.968050 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.969480 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.971004 (+ 1524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.971211 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.971276 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.971432 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57233,"thread_start_us":57106,"threads_started":1}
I20260812 06:54:20.972086 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.913555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:45607 (local address 127.0.0.1:33740)
0812 06:54:20.963629 (+ 50074us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:20.963648 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:20.963877 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:20.964825 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:20.964828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:20.964854 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:20.965217 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.965231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.967421 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.967425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.969332 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:20.969346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.971401 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.971406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:20.971564 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.971594 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:54:20.971679 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:54:20.971794 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49944,"thread_start_us":38860,"threads_started":1}
I20260812 06:54:20.972011 8653 tablet_service.cc:1467] Processing CreateTablet for tablet 6df6311b0b804f11834e872afc8ed17d (DEFAULT_TABLE table=test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:20.972807 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:20.914245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.17.69.67:45607)
0812 06:54:20.952378 (+ 38133us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:20.952388 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:20.963770 (+ 11382us) server_negotiation.cc:407] Connection header received
0812 06:54:20.964058 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:20.964067 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:20.964299 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:20.964454 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:20.965448 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.967244 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.969504 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:20.970893 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:20.970990 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:20.971209 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:20.972314 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0812 06:54:20.972464 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37940,"thread_start_us":39875,"threads_started":1}
I20260812 06:54:20.973685 8653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df6311b0b804f11834e872afc8ed17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:20.975479 8879 tablet_service.cc:1467] Processing CreateTablet for tablet 6df6311b0b804f11834e872afc8ed17d (DEFAULT_TABLE table=test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:20.976760 8879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df6311b0b804f11834e872afc8ed17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:20.984529 9118 tablet_service.cc:1467] Processing CreateTablet for tablet 6df6311b0b804f11834e872afc8ed17d (DEFAULT_TABLE table=test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:20.986881 9118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df6311b0b804f11834e872afc8ed17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:21.003161 9263 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Bootstrap starting.
Aug 12 06:54:21 dist-test-slave-6zbq krb5kdc[6699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for HTTP/127.17.69.84@KRBTEST.COM
I20260812 06:54:21.008751 9263 tablet_bootstrap.cc:654] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:21.010389 9263 log.cc:824] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:21.011686 9264 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Bootstrap starting.
I20260812 06:54:21.015277 9264 tablet_bootstrap.cc:654] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:21.015980 9266 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Bootstrap starting.
I20260812 06:54:21.019936 9264 log.cc:824] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:21.020569 9266 tablet_bootstrap.cc:654] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:21.021981 9266 log.cc:824] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:21.024130 9263 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: No bootstrap required, opened a new log
I20260812 06:54:21.024477 9263 ts_tablet_manager.cc:1397] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20260812 06:54:21.027348 9266 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: No bootstrap required, opened a new log
I20260812 06:54:21.027599 9266 ts_tablet_manager.cc:1397] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260812 06:54:21.037232 9264 tablet_bootstrap.cc:492] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: No bootstrap required, opened a new log
I20260812 06:54:21.037530 9264 ts_tablet_manager.cc:1397] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.005s
I20260812 06:54:21.037566 9266 raft_consensus.cc:348] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.038125 9266 raft_consensus.cc:374] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:21.038276 9266 raft_consensus.cc:729] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e47390691949df92b5ca1ace5ea8d6, State: Initialized, Role: FOLLOWER
I20260812 06:54:21.039078 9266 consensus_queue.cc:260] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.047181 9264 raft_consensus.cc:348] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.048331 9263 raft_consensus.cc:348] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.048786 9263 raft_consensus.cc:374] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:21.048916 9263 raft_consensus.cc:729] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b897aa7c5abf45d2b8c859b9eaae7c3c, State: Initialized, Role: FOLLOWER
I20260812 06:54:21.049681 9263 consensus_queue.cc:260] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.050318 9264 raft_consensus.cc:374] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:21.050475 9264 raft_consensus.cc:729] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0f6eaa01574982903fec84749e7c5e, State: Initialized, Role: FOLLOWER
I20260812 06:54:21.051162 9264 consensus_queue.cc:260] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.072158 9266 ts_tablet_manager.cc:1428] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Time spent starting tablet: real 0.044s user 0.030s sys 0.014s
I20260812 06:54:21.072728 8758 heartbeater.cc:503] Master 127.17.69.126:45195 was elected leader, sending a full tablet report...
I20260812 06:54:21.076749 9004 heartbeater.cc:503] Master 127.17.69.126:45195 was elected leader, sending a full tablet report...
I20260812 06:54:21.080135 9231 heartbeater.cc:503] Master 127.17.69.126:45195 was elected leader, sending a full tablet report...
I20260812 06:54:21.081434 9263 ts_tablet_manager.cc:1428] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Time spent starting tablet: real 0.057s user 0.011s sys 0.005s
W20260812 06:54:21.085803 9006 tablet.cc:2369] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:21.085876 9264 ts_tablet_manager.cc:1428] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Time spent starting tablet: real 0.048s user 0.013s sys 0.004s
I20260812 06:54:21.241218 9280 raft_consensus.cc:482] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:21.241499 9280 raft_consensus.cc:504] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.243239 9280 leader_election.cc:288] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56e47390691949df92b5ca1ace5ea8d6 (127.17.69.65:44089), ac0f6eaa01574982903fec84749e7c5e (127.17.69.67:45607)
I20260812 06:54:21.249832 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.243885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:57445 (local address 127.17.69.67:45607)
0812 06:54:21.244077 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.244085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.244360 (+ 275us) server_negotiation.cc:407] Connection header received
0812 06:54:21.244604 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.244609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.244685 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.244783 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:21.245401 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.246125 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.247958 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.248832 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.248982 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.249050 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.249396 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.249517 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:54:21.249845 9292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.243529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:45607 (local address 127.17.69.66:57445)
0812 06:54:21.244237 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:21.244257 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:21.244434 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:21.244826 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:21.244831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:21.244856 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:21.245244 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:21.245258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.246262 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.246267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:21.247815 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:21.247827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.249030 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.249036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:21.249214 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.249247 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:54:21.249352 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 06:54:21.249477 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":23,"thread_start_us":387,"threads_started":1}
I20260812 06:54:21.249887 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.243697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:36673 (local address 127.17.69.65:44089)
0812 06:54:21.244112 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.244118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.244590 (+ 472us) server_negotiation.cc:407] Connection header received
0812 06:54:21.244741 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.244744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.244828 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.244918 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:21.245521 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.246251 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.248021 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.248849 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.248987 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.249059 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.249532 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.249661 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260812 06:54:21.250531 9293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.243602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44089 (local address 127.17.69.66:36673)
0812 06:54:21.244505 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:21.244522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:21.244642 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:21.244945 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:21.244949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:21.244981 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:21.245345 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:21.245359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.246415 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.246420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:21.247864 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:21.247877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.249206 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.249213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:21.249367 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.249399 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:54:21.250085 (+ 686us) client_negotiation.cc:240] Negotiation successful
0812 06:54:21.250206 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":40,"thread_start_us":628,"threads_started":1}
I20260812 06:54:21.250757 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df6311b0b804f11834e872afc8ed17d" candidate_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac0f6eaa01574982903fec84749e7c5e" is_pre_election: true
I20260812 06:54:21.251197 9157 raft_consensus.cc:2454] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897aa7c5abf45d2b8c859b9eaae7c3c in term 0.
I20260812 06:54:21.251382 8677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df6311b0b804f11834e872afc8ed17d" candidate_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e47390691949df92b5ca1ace5ea8d6" is_pre_election: true
I20260812 06:54:21.251796 8808 leader_election.cc:302] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac0f6eaa01574982903fec84749e7c5e, b897aa7c5abf45d2b8c859b9eaae7c3c; no voters:
I20260812 06:54:21.251919 8677 raft_consensus.cc:2454] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b897aa7c5abf45d2b8c859b9eaae7c3c in term 0.
I20260812 06:54:21.252215 9280 raft_consensus.cc:2781] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:21.252368 9280 raft_consensus.cc:482] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:21.252462 9280 raft_consensus.cc:3037] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:21.254637 9280 raft_consensus.cc:504] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.255131 9280 leader_election.cc:288] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [CANDIDATE]: Term 1 election: Requested vote from peers 56e47390691949df92b5ca1ace5ea8d6 (127.17.69.65:44089), ac0f6eaa01574982903fec84749e7c5e (127.17.69.67:45607)
I20260812 06:54:21.255524 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df6311b0b804f11834e872afc8ed17d" candidate_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac0f6eaa01574982903fec84749e7c5e"
I20260812 06:54:21.255528 8677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df6311b0b804f11834e872afc8ed17d" candidate_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e47390691949df92b5ca1ace5ea8d6"
I20260812 06:54:21.255674 8677 raft_consensus.cc:3037] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:21.255690 9157 raft_consensus.cc:3037] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:21.257699 8677 raft_consensus.cc:2454] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897aa7c5abf45d2b8c859b9eaae7c3c in term 1.
I20260812 06:54:21.257712 9157 raft_consensus.cc:2454] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b897aa7c5abf45d2b8c859b9eaae7c3c in term 1.
I20260812 06:54:21.258059 8809 leader_election.cc:302] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e47390691949df92b5ca1ace5ea8d6, b897aa7c5abf45d2b8c859b9eaae7c3c; no voters:
I20260812 06:54:21.258359 9280 raft_consensus.cc:2781] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:21.258759 9280 raft_consensus.cc:686] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 1 LEADER]: Becoming Leader. State: Replica: b897aa7c5abf45d2b8c859b9eaae7c3c, State: Running, Role: LEADER
I20260812 06:54:21.259234 9280 consensus_queue.cc:237] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } }
I20260812 06:54:21.263314 8424 catalog_manager.cc:5495] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to b897aa7c5abf45d2b8c859b9eaae7c3c (127.17.69.66). New cstate: current_term: 1 leader_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b897aa7c5abf45d2b8c859b9eaae7c3c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 35301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 } health_report { overall_health: UNKNOWN } } }
W20260812 06:54:21.287000 8760 tablet.cc:2369] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:21.323877 9232 tablet.cc:2369] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:21.332304 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.327027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.17.69.66:35301)
0812 06:54:21.327207 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.327214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.327282 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:54:21.327510 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.327515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.327604 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.327737 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:21.328106 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.328859 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.330264 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.330706 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.330814 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.331287 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:21.331747 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:21.331872 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.331980 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.332056 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:54:21.339612 8677 raft_consensus.cc:1261] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 1 FOLLOWER]: Refusing update from remote peer b897aa7c5abf45d2b8c859b9eaae7c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:21.339609 9157 raft_consensus.cc:1261] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 1 FOLLOWER]: Refusing update from remote peer b897aa7c5abf45d2b8c859b9eaae7c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:21.340607 9295 consensus_queue.cc:1035] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac0f6eaa01574982903fec84749e7c5e" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 45607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:21.340929 9280 consensus_queue.cc:1035] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e47390691949df92b5ca1ace5ea8d6" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:21.353474 9304 mvcc.cc:205] Tried to move back new op lower bound from 7317576340837797888 to 7317576340521279488. Current Snapshot: MvccSnapshot[applied={T|T < 7317576340837797888}]
I20260812 06:54:21.355422 9307 mvcc.cc:205] Tried to move back new op lower bound from 7317576340837797888 to 7317576340521279488. Current Snapshot: MvccSnapshot[applied={T|T < 7317576340837797888}]
I20260812 06:54:21.365826 8424 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:21.366077 8424 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:21.369422 8424 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d852bd9053445b598a0c2456ad97d58: Sending DeleteTablet for 3 replicas of tablet 6df6311b0b804f11834e872afc8ed17d
I20260812 06:54:21.370446 8653 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df6311b0b804f11834e872afc8ed17d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:21 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:34248
I20260812 06:54:21.370458 8879 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df6311b0b804f11834e872afc8ed17d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:21 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:37744
I20260812 06:54:21.370725 9118 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df6311b0b804f11834e872afc8ed17d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:21 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:33740
I20260812 06:54:21.373648 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 8488
I20260812 06:54:21.383029 9322 tablet_replica.cc:330] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: stopping tablet replica
I20260812 06:54:21.383412 9322 raft_consensus.cc:2229] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:21.383849 9322 raft_consensus.cc:2258] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:21.385845 9322 ts_tablet_manager.cc:1905] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:21.387571 9318 tablet_replica.cc:330] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: stopping tablet replica
I20260812 06:54:21.388044 9318 raft_consensus.cc:2229] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:21.388695 9318 raft_consensus.cc:2258] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:21.389999 9317 tablet_replica.cc:330] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: stopping tablet replica
I20260812 06:54:21.390420 9317 raft_consensus.cc:2229] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:21.390955 9317 raft_consensus.cc:2258] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:21.392557 9318 ts_tablet_manager.cc:1905] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:21.392930 9317 ts_tablet_manager.cc:1905] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:21.395063 9322 ts_tablet_manager.cc:1918] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:21.398048 9322 log.cc:1196] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/6df6311b0b804f11834e872afc8ed17d
I20260812 06:54:21.399658 9317 ts_tablet_manager.cc:1918] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:21.399890 9317 log.cc:1196] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/6df6311b0b804f11834e872afc8ed17d
I20260812 06:54:21.400516 9317 ts_tablet_manager.cc:1939] T 6df6311b0b804f11834e872afc8ed17d P 56e47390691949df92b5ca1ace5ea8d6: Deleting consensus metadata
I20260812 06:54:21.402232 9322 ts_tablet_manager.cc:1939] T 6df6311b0b804f11834e872afc8ed17d P ac0f6eaa01574982903fec84749e7c5e: Deleting consensus metadata
I20260812 06:54:21.403012 8401 catalog_manager.cc:4841] TS 56e47390691949df92b5ca1ace5ea8d6 (127.17.69.65:44089): tablet 6df6311b0b804f11834e872afc8ed17d (table test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]) successfully deleted
I20260812 06:54:21.403851 8400 catalog_manager.cc:4841] TS ac0f6eaa01574982903fec84749e7c5e (127.17.69.67:45607): tablet 6df6311b0b804f11834e872afc8ed17d (table test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]) successfully deleted
I20260812 06:54:21.404896 9318 ts_tablet_manager.cc:1918] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:21.405057 9318 log.cc:1196] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/6df6311b0b804f11834e872afc8ed17d
I20260812 06:54:21.405578 9318 ts_tablet_manager.cc:1939] T 6df6311b0b804f11834e872afc8ed17d P b897aa7c5abf45d2b8c859b9eaae7c3c: Deleting consensus metadata
I20260812 06:54:21.405730 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.374786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.17.69.65:44089)
0812 06:54:21.374943 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.374949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.389431 (+ 14482us) server_negotiation.cc:407] Connection header received
0812 06:54:21.389888 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.389894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.389970 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.390085 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:21.392157 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.393224 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.395041 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.395667 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.395768 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.399185 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:21.399198 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:21.399212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:21.399249 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:21.402787 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.403836 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.403845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.403848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.404001 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.404392 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.404396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.404398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.404734 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:21.405078 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.405305 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.405454 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:54:21.406863 8751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:21.407073 8406 catalog_manager.cc:4841] TS b897aa7c5abf45d2b8c859b9eaae7c3c (127.17.69.66:35301): tablet 6df6311b0b804f11834e872afc8ed17d (table test-table [id=1c85bba4b19d4888b12f3cdb8ead09ce]) successfully deleted
I20260812 06:54:21.493649 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 8488
I20260812 06:54:21.519098 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 8764
I20260812 06:54:21.532968 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.519795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.17.69.66:35301)
0812 06:54:21.520014 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.520020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.520040 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:54:21.520230 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.520234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.520297 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.520382 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:21.521757 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.522734 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.524098 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.524553 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.524671 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.527102 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:21.527114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:21.527129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:21.527164 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:21.530093 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.530920 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.530927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.530929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.531077 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.531440 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.531443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.531445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.531804 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:21.532116 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.532399 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.532607 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 06:54:21.533620 8988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:21.641768 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 8764
I20260812 06:54:21.667295 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 9011
I20260812 06:54:21.680037 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:21.668019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.17.69.67:45607)
0812 06:54:21.668193 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:21.668199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:21.668219 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:54:21.668416 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:21.668421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:21.668496 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:21.668578 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:21.669901 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.670661 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.671976 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:21.672376 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:21.672487 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:21.675012 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:21.675025 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:21.675040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:21.675077 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:21.677419 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.678050 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.678057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.678060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.678196 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:21.678578 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:21.678583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:21.678585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:21.678950 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:21.679216 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:21.679515 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 06:54:21.679730 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:54:21.680621 9223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:21.766209 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 9011
I20260812 06:54:21.786840 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 8365
I20260812 06:54:21.788033 8479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:21.895505 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 8365
I20260812 06:54:21.926580 17685 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 06:54:22.270371 17685 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 06:54:22.270569 17685 mini_ranger.cc:66] Stopping Ranger...
I20260812 06:54:22.614797 17685 mini_ranger.cc:68] Stopped Ranger
2026-08-12 06:54:22.615 UTC [6744] LOG: received smart shutdown request
2026-08-12 06:54:22.617 UTC [6744] LOG: background worker "logical replication launcher" (PID 6762) exited with exit code 1
2026-08-12 06:54:22.617 UTC [6753] LOG: shutting down
2026-08-12 06:54:22.750 UTC [6744] LOG: database system is shut down
2026-08-12T06:54:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49539 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:22 dist-test-slave-6zbq krb5kdc[9351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:22 dist-test-slave-6zbq krb5kdc[9351](info): set up 2 sockets
Aug 12 06:54:22 dist-test-slave-6zbq krb5kdc[9351](info): commencing operation
krb5kdc: starting...
W20260812 06:54:24.111078 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:54:24 dist-test-slave-6zbq krb5kdc[9351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:54:24.344846 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41971
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:24.660692 9377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:24.660964 9377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:24.661115 9377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:24.668021 9377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:24.668120 9377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:24.668197 9377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:24.668275 9377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:24.668335 9377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:24.677562 9377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:24.678756 9377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:24.679924 9377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:24.690689 9384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:24.690647 9394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:24.690733 9390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:24.691710 9377 server_base.cc:1034] running on GCE node
I20260812 06:54:24.692382 9377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:24.693578 9377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:24.694764 9377 hybrid_clock.cc:648] HybridClock initialized: now 1786517664694722 us; error 33 us; skew 500 ppm
I20260812 06:54:24.695066 9377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:24 dist-test-slave-6zbq krb5kdc[9351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:24.697883 9377 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:24.699177 9377 webserver.cc:460] Webserver started at http://127.17.69.126:44255/ using document root <none> and password file <none>
I20260812 06:54:24.699767 9377 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:24.699882 9377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:24.700140 9377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:24.702913 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "d79b65f608bd4bc48c9fae1bed7a29a2"
format_stamp: "Formatted at 2026-08-12 06:54:24 on dist-test-slave-6zbq"
server_key: "3ab61bcc8ccb59f438776b4288b83610"
server_key_iv: "d237641f312df02039af990c7107c7ef"
server_key_version: "encryptionkey@0"
I20260812 06:54:24.703658 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "d79b65f608bd4bc48c9fae1bed7a29a2"
format_stamp: "Formatted at 2026-08-12 06:54:24 on dist-test-slave-6zbq"
server_key: "3ab61bcc8ccb59f438776b4288b83610"
server_key_iv: "d237641f312df02039af990c7107c7ef"
server_key_version: "encryptionkey@0"
I20260812 06:54:24.708258 9377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:24.711290 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:24.712443 9377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:24.712585 9377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "d79b65f608bd4bc48c9fae1bed7a29a2"
format_stamp: "Formatted at 2026-08-12 06:54:24 on dist-test-slave-6zbq"
server_key: "3ab61bcc8ccb59f438776b4288b83610"
server_key_iv: "d237641f312df02039af990c7107c7ef"
server_key_version: "encryptionkey@0"
I20260812 06:54:24.712749 9377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:24.733143 9377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:24.736927 9377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:24.737159 9377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:24.753160 9377 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:41971
I20260812 06:54:24.753221 9484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:41971 every 8 connection(s)
I20260812 06:54:24.754283 9377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:24.757616 9485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:24.763357 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 9377
I20260812 06:54:24.763558 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:54:24.764185 17685 external_mini_cluster.cc:1428] Setting key 109c31e6a6e173de125d4168a2921c3a
I20260812 06:54:24.763782 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:54:24 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:24.784643 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.767484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.17.69.126:41971)
0812 06:54:24.768062 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:24.768068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:24.768111 (+ 43us) server_negotiation.cc:407] Connection header received
0812 06:54:24.769754 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:24.769759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:24.769957 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:24.770103 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:24.771478 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:24.772629 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:24.774383 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:24.774922 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:24.775064 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:24.778569 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:24.778577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:24.778586 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:24.778613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:24.780974 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:24.781771 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:24.781777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:24.781779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:24.781913 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:24.782317 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:24.782320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:24.782321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:24.782756 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:24.782898 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:24.783454 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 06:54:24.783763 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":423,"thread_start_us":284,"threads_started":1}
I20260812 06:54:24.784670 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.766511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41971 (local address 127.0.0.1:40944)
0812 06:54:24.767229 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.767278 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:24.769422 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:24.770166 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:24.770177 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:24.770866 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:24.771287 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:24.771310 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:24.772775 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:24.772780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:24.774204 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:24.774218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:24.775147 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:24.775158 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:24.775323 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:24.775955 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:24.775976 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:24.778360 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:24.781120 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:24.781134 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:24.781144 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:24.781647 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:24.782014 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:24.782018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:24.782021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:24.782219 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:24.782891 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:24.782898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:24.783104 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 06:54:24.783366 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 06:54:24.783899 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":102,"thread_start_us":287,"threads_started":1}
I20260812 06:54:24.787534 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:54:24.788551 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.788224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48002)
0812 06:54:24.788365 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.788459 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 06:54:24.788861 9411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
I20260812 06:54:24.790246 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.789955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48006)
0812 06:54:24.790076 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.790139 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:54:24.790462 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:40307: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.69.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
I20260812 06:54:24.820711 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:40307
--webserver_interface=127.17.69.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:24.839802 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } attempt: 1
I20260812 06:54:24.840869 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.840459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48010)
0812 06:54:24.840634 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.840734 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 06:54:24.841830 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.841488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48024)
0812 06:54:24.841630 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.841697 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:54:24.842008 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:40307: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
I20260812 06:54:24.910238 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } attempt: 2
I20260812 06:54:24.911322 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.910866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48026)
0812 06:54:24.911096 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.911179 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:54:24.912328 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:24.911984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48030)
0812 06:54:24.912122 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:24.912198 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:54:24.912508 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:40307: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
I20260812 06:54:25.003729 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } attempt: 3
I20260812 06:54:25.004918 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.004482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48032)
0812 06:54:25.004681 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.004788 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 06:54:25.005921 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.005570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48042)
0812 06:54:25.005717 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.005785 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:54:25.006104 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:40307: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
W20260812 06:54:25.131527 9493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:25.131790 9493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:25.131925 9493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:25.138753 9493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:25.138857 9493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:25.138934 9493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:25.139010 9493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:25.139076 9493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:25.148214 9493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:40307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.69.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:25.149374 9493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:25.149348 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } attempt: 4
I20260812 06:54:25.150475 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.149904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48056)
0812 06:54:25.150212 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.150298 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260812 06:54:25.150583 9493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 06:54:25.151554 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.151169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48062)
0812 06:54:25.151344 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.151436 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 06:54:25.151777 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:40307: Network error: Client connection negotiation failed: client connection to 127.17.69.125:40307: connect: Connection refused (error 111)
W20260812 06:54:25.161486 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:25.161564 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:25.161665 9502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:25.162499 9493 server_base.cc:1034] running on GCE node
I20260812 06:54:25.163197 9493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:25.164422 9493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:25.165611 9493 hybrid_clock.cc:648] HybridClock initialized: now 1786517665165563 us; error 44 us; skew 500 ppm
I20260812 06:54:25.165904 9493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:25.168808 9493 init.cc:377] Logged in from keytab as kudu/127.17.69.125@KRBTEST.COM (short username kudu)
I20260812 06:54:25.170089 9493 webserver.cc:460] Webserver started at http://127.17.69.125:39439/ using document root <none> and password file <none>
I20260812 06:54:25.170722 9493 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:25.170819 9493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:25.171051 9493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:25.173723 9493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data/instance:
uuid: "f7a306ff715046fb9b6fb4185634f28d"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "539344073bf904eafc6cfef9a6056d4c"
server_key_iv: "4f6f31eba01237ec1556f0d890139a1b"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.174523 9493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal/instance:
uuid: "f7a306ff715046fb9b6fb4185634f28d"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "539344073bf904eafc6cfef9a6056d4c"
server_key_iv: "4f6f31eba01237ec1556f0d890139a1b"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.178984 9493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:54:25.181975 9521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:25.183113 9493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:54:25.183272 9493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "f7a306ff715046fb9b6fb4185634f28d"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "539344073bf904eafc6cfef9a6056d4c"
server_key_iv: "4f6f31eba01237ec1556f0d890139a1b"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.183439 9493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:25.208433 9493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:25.212229 9493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:25.212466 9493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:25.229259 9493 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.125:40307
I20260812 06:54:25.229295 9610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.125:40307 every 8 connection(s)
I20260812 06:54:25.230444 9493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
I20260812 06:54:25.233660 9613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:25.239514 9613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:54:25.240520 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 9493
I20260812 06:54:25.240717 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-1/wal/instance
I20260812 06:54:25.241143 17685 external_mini_cluster.cc:1428] Setting key 79b96e2d11d32ec0d646d4d38c2f4766
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:25.258412 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.241850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.17.69.126:41971)
0812 06:54:25.242045 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.242051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.243105 (+ 1054us) server_negotiation.cc:407] Connection header received
0812 06:54:25.244833 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.244838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.244916 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.245020 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.246291 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.247223 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.248647 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.249209 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.249333 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.252498 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.252504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.252509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.252548 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.255586 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.256263 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.256268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.256271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.256411 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.256774 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.256779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.256782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.257117 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.257259 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.257872 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.258085 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:54:25.259050 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.242218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41971 (local address 127.0.0.1:40960)
0812 06:54:25.242947 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.242988 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.244590 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.245172 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.245179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.245632 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.246118 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.246133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.247354 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.247358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.248498 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.248512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.249440 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.249446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.249633 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.250253 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.250279 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.252278 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.255728 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.255734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.255744 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.256149 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.256504 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.256507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.256509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.256667 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.257235 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.257240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.257528 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.257741 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.258066 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":315,"threads_started":1}
I20260812 06:54:25.260604 9613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.69.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.124@KRBTEST.COM" created.
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:54:25.275713 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.261314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48068)
0812 06:54:25.261489 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.261509 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.261624 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.262462 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.262466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.262853 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.262973 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.262980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.264237 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.264240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.265027 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.265037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.265770 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.265775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.265968 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.266439 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.266455 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.270123 (+ 3668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.273570 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.273573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.273576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.273916 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.274301 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.274304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.274306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.274460 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.275074 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.275077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.275202 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.275303 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.275473 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 06:54:25.276026 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.261336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.17.69.125:40307)
0812 06:54:25.262098 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.262104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.262126 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:54:25.262164 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.262169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.262310 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.262459 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.263116 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.264110 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.265150 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.265640 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.265754 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.270360 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.270375 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.270391 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.270432 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.273451 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.274041 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.274049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.274052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.274193 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.274556 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.274561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.274564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.274963 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.275164 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.275455 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.275682 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":498,"threads_started":1}
I20260812 06:54:25.278185 9613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:54:25.279214 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.278725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40330)
0812 06:54:25.278893 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.279012 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:54:25.279487 9525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.280913 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.280562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40336)
0812 06:54:25.280699 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.280790 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:54:25.281191 9613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
I20260812 06:54:25.297494 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:46551
--webserver_interface=127.17.69.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:25.330461 9613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 1
I20260812 06:54:25.331662 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.331140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40338)
0812 06:54:25.331353 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.331482 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:54:25.332626 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.332251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40352)
0812 06:54:25.332396 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.332494 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:54:25.332835 9613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.401058 9613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 2
I20260812 06:54:25.402238 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.401729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40366)
0812 06:54:25.401944 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.402054 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 06:54:25.403211 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.402857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40368)
0812 06:54:25.402993 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.403077 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 06:54:25.403411 9613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.451041 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } attempt: 5
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:54:25.465258 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.451798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:48076)
0812 06:54:25.452114 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.452134 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.452269 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.452841 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.452844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.453209 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.453392 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.453400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.454766 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.454769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.455632 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.455657 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.456604 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.456607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.456751 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.457347 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.457364 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.459513 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.462974 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.462978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.462981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.463342 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.463695 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.463698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.463700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.463821 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.464462 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.464466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.464655 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.464782 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.464988 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260812 06:54:25.465441 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.452069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.17.69.125:40307)
0812 06:54:25.452433 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.452440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.452464 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:54:25.452554 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.452559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.452658 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.452777 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.453557 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.454636 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.455792 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.456475 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.456604 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.459702 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.459709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.459714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.459751 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.462852 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.463452 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.463457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.463461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.463602 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.463915 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.463921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.463924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.464317 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.464462 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.464907 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.465133 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260812 06:54:25.466500 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:54:25.467499 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.467153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40372)
0812 06:54:25.467303 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.467374 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:54:25.468397 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.468116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40384)
0812 06:54:25.468237 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.468312 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 06:54:25.468621 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.494642 9613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 3
I20260812 06:54:25.495703 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.495210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40398)
0812 06:54:25.495420 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.495524 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 06:54:25.496763 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.496311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40408)
0812 06:54:25.496532 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.496625 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260812 06:54:25.496965 9613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.519820 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 1
I20260812 06:54:25.520805 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.520398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40414)
0812 06:54:25.520561 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.520651 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 06:54:25.521665 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.521388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40430)
0812 06:54:25.521499 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.521565 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 06:54:25.521867 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.590080 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 2
I20260812 06:54:25.591260 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.590797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40444)
0812 06:54:25.591004 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.591095 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 06:54:25.592262 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.591938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40448)
0812 06:54:25.592089 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.592158 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:54:25.592461 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
W20260812 06:54:25.622695 9632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:25.622951 9632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:25.623082 9632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:25.630021 9632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:25.630108 9632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:25.630178 9632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:25.630244 9632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:25.630299 9632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:25.639499 9632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41971,127.17.69.125:40307,127.17.69.124:46551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:46551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.69.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:25.640172 9613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 4
I20260812 06:54:25.640836 9632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:25.641346 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.640775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40450)
0812 06:54:25.641029 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.641153 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 06:54:25.642408 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.642005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40452)
0812 06:54:25.642153 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.642241 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:54:25.642504 9632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:25.642623 9613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
W20260812 06:54:25.653236 9640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:25.653266 9646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:25.653309 9642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:25.654219 9632 server_base.cc:1034] running on GCE node
I20260812 06:54:25.654903 9632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:25.656028 9632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:25.657215 9632 hybrid_clock.cc:648] HybridClock initialized: now 1786517665657164 us; error 41 us; skew 500 ppm
I20260812 06:54:25.657490 9632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:25.660359 9632 init.cc:377] Logged in from keytab as kudu/127.17.69.124@KRBTEST.COM (short username kudu)
I20260812 06:54:25.661731 9632 webserver.cc:460] Webserver started at http://127.17.69.124:45057/ using document root <none> and password file <none>
I20260812 06:54:25.662333 9632 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:25.662494 9632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:25.662741 9632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:25.665457 9632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data/instance:
uuid: "37b03828e0ea421289c08d7f78baae41"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "12bfe3d649ea84f70b386aed37dff7fa"
server_key_iv: "df3afff3fb66c9eaaf6d13dfe35dac8f"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.666188 9632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal/instance:
uuid: "37b03828e0ea421289c08d7f78baae41"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "12bfe3d649ea84f70b386aed37dff7fa"
server_key_iv: "df3afff3fb66c9eaaf6d13dfe35dac8f"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.671106 9632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:54:25.674271 9657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:25.675478 9632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:54:25.675626 9632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "37b03828e0ea421289c08d7f78baae41"
format_stamp: "Formatted at 2026-08-12 06:54:25 on dist-test-slave-6zbq"
server_key: "12bfe3d649ea84f70b386aed37dff7fa"
server_key_iv: "df3afff3fb66c9eaaf6d13dfe35dac8f"
server_key_version: "encryptionkey@0"
I20260812 06:54:25.675807 9632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:25.688271 9632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:25.692066 9632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:25.692303 9632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:25.698644 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 3
I20260812 06:54:25.699757 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.699245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40460)
0812 06:54:25.699457 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.699567 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 06:54:25.700803 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.700368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:40472)
0812 06:54:25.700561 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.700665 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:54:25.701056 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46551: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46551: connect: Connection refused (error 111)
I20260812 06:54:25.709299 9632 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.124:46551
I20260812 06:54:25.709352 9733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.124:46551 every 8 connection(s)
I20260812 06:54:25.710520 9632 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
I20260812 06:54:25.713825 9734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:25.718211 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 9632
I20260812 06:54:25.718451 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786517466184829-17685-0/minicluster-data/master-2/wal/instance
I20260812 06:54:25.718986 17685 external_mini_cluster.cc:1428] Setting key 3895c9fc63c0aedd211240c71df5ddd0
I20260812 06:54:25.720898 9734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:54:25.738309 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.723303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.17.69.126:41971)
0812 06:54:25.723493 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.723499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.724129 (+ 630us) server_negotiation.cc:407] Connection header received
0812 06:54:25.724378 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.724382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.724443 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.724522 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.725728 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.726443 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.728273 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.729033 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.729172 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.732546 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.732551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.732555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.732587 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.735312 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.736204 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.736208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.736210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.736335 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.736776 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.736779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.736781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.737035 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.737202 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.737808 (+ 606us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.738022 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:54:25.739317 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.723270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41971 (local address 127.0.0.1:40972)
0812 06:54:25.723998 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.724042 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.724247 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.724606 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.724613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.725148 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.725563 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.725584 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.726602 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.726606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.728100 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.728116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.729291 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.729300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.729552 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.730235 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.730262 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.732362 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.735432 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.735444 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.735455 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.736085 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.736436 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.736441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.736445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.736681 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.737160 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.737166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.737435 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.737699 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.738042 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":34,"thread_start_us":348,"threads_started":1}
I20260812 06:54:25.739799 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.720672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.17.69.124:46551)
0812 06:54:25.721459 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.721472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.721511 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:54:25.723241 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.723265 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.723656 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.723992 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.725403 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.726564 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.728324 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.729090 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.729239 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.733785 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.733793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.733803 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.733830 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.736898 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.737565 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.737572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.737575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.737718 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.738121 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.738125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.738128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.738586 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.738853 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.739191 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.739413 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":583,"thread_start_us":395,"threads_started":1}
I20260812 06:54:25.740911 9734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:25.753774 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.741676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:40307 (local address 127.0.0.1:54858)
0812 06:54:25.741833 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.741851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.741953 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.742248 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.742252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.742662 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.742846 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.742857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.743861 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.743866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.744825 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.744838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.745553 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.745560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.745739 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.746209 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.746231 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.748274 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.751244 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.751249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.751253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.751670 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.752057 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.752060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.752062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.752226 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.752902 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.752907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.753059 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.753209 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.753449 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:54:25.754184 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.741636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.17.69.125:40307)
0812 06:54:25.741838 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.741844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.741919 (+ 75us) server_negotiation.cc:407] Connection header received
0812 06:54:25.742053 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.742057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.742117 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.742193 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.742989 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.743719 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.744961 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.745415 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.745503 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.748446 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.748450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.748453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.748484 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.751109 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.751791 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.751797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.751801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.751963 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.752333 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.752338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.752342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.752674 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.752834 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.753328 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.753546 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:54:25.755100 9734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:54:25.756804 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.742552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.17.69.126:41971)
0812 06:54:25.742694 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.742698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.742774 (+ 76us) server_negotiation.cc:407] Connection header received
0812 06:54:25.742927 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.742931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.742995 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.743091 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.744203 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.744900 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.746442 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.746838 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.746929 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.750695 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.750701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.750706 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.750744 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.753931 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.754720 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.754725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.754729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.754892 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.755261 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.755266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.755270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.755547 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.755811 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.756264 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.756469 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:54:25.768139 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.755677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:54702)
0812 06:54:25.755919 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.755931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.756019 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.756389 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.756393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.756981 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.757168 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.757180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.758190 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.758195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.759395 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.759407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.760044 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.760050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.760217 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.761453 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.761469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.763287 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.765967 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.765973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.765976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.766320 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.766682 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.766686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.766690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.766809 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.767259 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.767263 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.767388 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.767501 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.767785 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260812 06:54:25.768218 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.755720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.17.69.124:46551)
0812 06:54:25.755859 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.755863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.755992 (+ 129us) server_negotiation.cc:407] Connection header received
0812 06:54:25.756099 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.756103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.756161 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.756237 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.757326 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.758081 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.759513 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.759919 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.760019 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.763465 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.763471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.763475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.763508 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.765835 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.766456 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.766461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.766465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.766583 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.766913 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.766917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.766919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.767153 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.767274 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.767612 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.767939 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":51}
I20260812 06:54:25.772404 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.758788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.17.69.125:40307)
0812 06:54:25.758938 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.758944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.758959 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:54:25.759001 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.759006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.759066 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.759154 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.760597 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.761273 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.762929 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.763316 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.763432 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.767257 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.767263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.767267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.767295 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.770024 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.770710 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.770716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.770719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.770869 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.771179 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.771184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.771187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.771499 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.771755 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.771908 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.772116 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:54:25.773268 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: Bootstrap starting.
I20260812 06:54:25.776371 9734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:25.776623 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:54:25.777333 9734 log.cc:824] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:25.779883 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: No bootstrap required, opened a new log
I20260812 06:54:25.789716 9734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.790011 9734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:25.790131 9734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b03828e0ea421289c08d7f78baae41, State: Initialized, Role: FOLLOWER
I20260812 06:54:25.790748 9734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.791728 9752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:25.792392 9752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:25.792917 9734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:25.802915 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.781709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.17.69.126:41971)
0812 06:54:25.782033 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.782039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.782056 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:54:25.782209 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.782214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.782277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.782371 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.783906 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.784591 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.785999 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.786471 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.786582 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.794455 (+ 7873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.794462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.794466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.794504 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.798073 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.799999 (+ 1926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.800006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.800009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.800196 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.800631 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.800637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.800640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.801288 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.801521 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.802070 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.802315 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260812 06:54:25.804301 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.787621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.17.69.124:46551)
0812 06:54:25.787876 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.787880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.788561 (+ 681us) server_negotiation.cc:407] Connection header received
0812 06:54:25.788719 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.788724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.788795 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.788891 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.790555 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.791603 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.793424 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.794170 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.794311 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.797370 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.797377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.797382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.797422 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.801003 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.801714 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.801719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.801723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.801891 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.802333 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.802358 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.802362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.802768 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.803008 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.803691 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.803905 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260812 06:54:25.807685 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.781404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.17.69.125:40307)
0812 06:54:25.781551 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.781557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.787315 (+ 5758us) server_negotiation.cc:407] Connection header received
0812 06:54:25.787519 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.787524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.787578 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.787670 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.791707 (+ 4037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.792693 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.795090 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.795583 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.795654 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.800276 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.800283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.800287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.800323 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.804830 (+ 4507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.805568 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.805574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.805577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.805744 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.806200 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.806206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.806209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.806582 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.806817 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.807164 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.807378 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 06:54:25.812477 9767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:54:25.812587 9767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:54:25.810212 9632 master_runner.cc:186] Error getting master registration for 127.17.69.126:41971: OK, instance_id { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" instance_seqno: 1786517664748687 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:54:25.821161 9632 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:54:25.851670 9752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:25.851902 9752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.853116 9752 leader_election.cc:288] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d79b65f608bd4bc48c9fae1bed7a29a2 (127.17.69.126:41971), f7a306ff715046fb9b6fb4185634f28d (127.17.69.125:40307)
I20260812 06:54:25.878259 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 4
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:54:25.893620 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.878824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:54732)
0812 06:54:25.879038 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.879056 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.879169 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.879756 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.879760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.880246 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.880416 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.880428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.881884 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.881890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.883383 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.883399 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.884315 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.884321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.884513 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.885100 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.885118 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.887516 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.891281 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.891287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.891291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.891611 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.891998 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.892003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.892007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.892134 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.892702 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.892706 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.892875 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.892990 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.893243 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 06:54:25.893674 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.878873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.17.69.124:46551)
0812 06:54:25.879037 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.879043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.879278 (+ 235us) server_negotiation.cc:407] Connection header received
0812 06:54:25.879348 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.879352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.879469 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.879577 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.880569 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.881749 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.883515 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.884184 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.884300 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.887690 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.887700 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.887705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.887740 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.891164 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.891719 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.891725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.891729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.891896 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.892236 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.892242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.892245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.892577 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.892709 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.893110 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.893342 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:54:25.899760 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: Bootstrap starting.
I20260812 06:54:25.903674 9485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:25.904966 9485 log.cc:824] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:25.908068 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: No bootstrap required, opened a new log
I20260812 06:54:25.918777 9485 raft_consensus.cc:348] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.919130 9485 raft_consensus.cc:374] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:25.919245 9485 raft_consensus.cc:729] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79b65f608bd4bc48c9fae1bed7a29a2, State: Initialized, Role: FOLLOWER
I20260812 06:54:25.919840 9485 consensus_queue.cc:260] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.920861 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:25.921277 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:25.922060 9485 sys_catalog.cc:564] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:25.922117 9446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37b03828e0ea421289c08d7f78baae41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" is_pre_election: true
I20260812 06:54:25.922652 9446 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37b03828e0ea421289c08d7f78baae41 in term 0.
I20260812 06:54:25.923233 9660 leader_election.cc:302] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b03828e0ea421289c08d7f78baae41, d79b65f608bd4bc48c9fae1bed7a29a2; no voters:
I20260812 06:54:25.923703 9752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:25.923878 9752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:25.923985 9752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:25.927104 9752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.927795 9752 leader_election.cc:288] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [CANDIDATE]: Term 1 election: Requested vote from peers d79b65f608bd4bc48c9fae1bed7a29a2 (127.17.69.126:41971), f7a306ff715046fb9b6fb4185634f28d (127.17.69.125:40307)
I20260812 06:54:25.929198 9446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37b03828e0ea421289c08d7f78baae41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2"
I20260812 06:54:25.929631 9446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:25.933609 9446 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37b03828e0ea421289c08d7f78baae41 in term 1.
I20260812 06:54:25.934064 9660 leader_election.cc:302] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b03828e0ea421289c08d7f78baae41, d79b65f608bd4bc48c9fae1bed7a29a2; no voters:
I20260812 06:54:25.934370 9752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:25.934841 9752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [term 1 LEADER]: Becoming Leader. State: Replica: 37b03828e0ea421289c08d7f78baae41, State: Running, Role: LEADER
I20260812 06:54:25.935333 9752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:25.938369 9788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37b03828e0ea421289c08d7f78baae41. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:25.938732 9788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:25.939986 9804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:25.941834 9613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } attempt: 5
W20260812 06:54:25.944092 9665 tablet.cc:2369] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:25.952463 9805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:54:25.952728 9805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 06:54:25 dist-test-slave-6zbq krb5kdc[9351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:54:25.967839 9804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:25.952623 9377 master_runner.cc:186] Error getting master registration for 127.17.69.125:40307: OK, instance_id { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" instance_seqno: 1786517665224519 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:54:25.968525 9377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:54:25.970413 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.942418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.17.69.124:46551)
0812 06:54:25.942572 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:25.942578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:25.942672 (+ 94us) server_negotiation.cc:407] Connection header received
0812 06:54:25.942853 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:25.942858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:25.942927 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:25.943025 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:25.944188 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.945333 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.948447 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.949115 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.949229 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.962313 (+ 13084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:25.962321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:25.962325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:25.962378 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:25.965786 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.966592 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.966597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.966601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.966734 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:25.967141 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:25.967146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:25.967149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:25.967514 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:25.967665 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:25.968843 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0812 06:54:25.969371 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 06:54:25.970695 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:25.942388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46551 (local address 127.0.0.1:54748)
0812 06:54:25.942589 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:25.942607 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:25.942719 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:25.943056 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:25.943061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:25.943586 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:25.943790 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.943803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.946824 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.946830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.948150 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:25.948166 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:25.958485 (+ 10319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:25.958491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:25.958727 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:25.959357 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:25.959382 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:25.962077 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:25.965936 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.965941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.965945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.966459 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.966855 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:25.966860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:25.966864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:25.967017 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:25.967655 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:25.967697 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:25.968263 (+ 566us) client_negotiation.cc:769] Sending connection context
0812 06:54:25.969523 (+ 1260us) client_negotiation.cc:240] Negotiation successful
0812 06:54:25.970257 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 06:54:25.976024 9613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: Bootstrap starting.
I20260812 06:54:25.981485 9613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:25.982820 9613 log.cc:824] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: Log is configured to *not* fsync() on all Append() calls
W20260812 06:54:25.989925 9413 tablet.cc:2369] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:25.992650 9613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: No bootstrap required, opened a new log
I20260812 06:54:25.993422 9446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [term 1 FOLLOWER]: Refusing update from remote peer 37b03828e0ea421289c08d7f78baae41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:25.994398 9788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:26.003301 9613 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:26.003672 9613 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:26.003809 9613 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a306ff715046fb9b6fb4185634f28d, State: Initialized, Role: FOLLOWER
I20260812 06:54:26.004446 9613 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } }
I20260812 06:54:26.013383 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37b03828e0ea421289c08d7f78baae41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a306ff715046fb9b6fb4185634f28d"
I20260812 06:54:26.013337 9569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37b03828e0ea421289c08d7f78baae41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a306ff715046fb9b6fb4185634f28d" is_pre_election: true
I20260812 06:54:26.013664 9566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:26.013312 9613 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:26.011335 9818 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.015367 9818 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:26.017321 9566 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [term 1 FOLLOWER]: Refusing update from remote peer 37b03828e0ea421289c08d7f78baae41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:26.018080 9788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:26.027757 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37b03828e0ea421289c08d7f78baae41. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.028101 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:26.031018 9807 mvcc.cc:205] Tried to move back new op lower bound from 7317576359895912448 to 7317576359675936768. Current Snapshot: MvccSnapshot[applied={T|T < 7317576359895912448}]
I20260812 06:54:26.034096 9815 mvcc.cc:205] Tried to move back new op lower bound from 7317576359895912448 to 7317576359675936768. Current Snapshot: MvccSnapshot[applied={T|T < 7317576359895912448}]
I20260812 06:54:26.037912 9818 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37b03828e0ea421289c08d7f78baae41. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.038235 9818 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:26.038749 9818 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.038951 9818 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:26.041188 9752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.041522 9752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:26.041754 9752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.041976 9752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:26.042719 9804 catalog_manager.cc:1347] Generated new cluster ID: 630ae3b8e3a54b508ff0d682db6f6d95
I20260812 06:54:26.042824 9804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:26.051272 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37b03828e0ea421289c08d7f78baae41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79b65f608bd4bc48c9fae1bed7a29a2" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41971 } } peers { permanent_uuid: "f7a306ff715046fb9b6fb4185634f28d" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 40307 } } peers { permanent_uuid: "37b03828e0ea421289c08d7f78baae41" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46551 } } }
I20260812 06:54:26.051492 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:54:26.063102 9804 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:26.063728 9821 mvcc.cc:205] Tried to move back new op lower bound from 7317576359895912448 to 7317576359675936768. Current Snapshot: MvccSnapshot[applied={T|T < 7317576359895912448 or (T in {7317576359895912448})}]
I20260812 06:54:26.065742 9804 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:26.085475 9804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37b03828e0ea421289c08d7f78baae41: Generated new TSK 0
I20260812 06:54:26.088774 9804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:54:26.110163 9842 catalog_manager.cc:1259] Loaded cluster ID: 630ae3b8e3a54b508ff0d682db6f6d95
I20260812 06:54:26.110260 9842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: loading cluster ID for follower catalog manager: success
I20260812 06:54:26.113813 9842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: acquiring CA information for follower catalog manager: success
I20260812 06:54:26.124724 9842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f7a306ff715046fb9b6fb4185634f28d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:54:26.954725 9805 catalog_manager.cc:1259] Loaded cluster ID: 630ae3b8e3a54b508ff0d682db6f6d95
I20260812 06:54:26.954818 9805 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: loading cluster ID for follower catalog manager: success
I20260812 06:54:26.957202 9805 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: acquiring CA information for follower catalog manager: success
I20260812 06:54:26.958941 9805 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d79b65f608bd4bc48c9fae1bed7a29a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:54:27.223966 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 9377
I20260812 06:54:27.225093 9479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:27.336692 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 9377
I20260812 06:54:27.362586 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 9493
I20260812 06:54:27.363718 9606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:27.466009 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 9493
I20260812 06:54:27.491219 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 9632
I20260812 06:54:27.492298 9727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:27.597388 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 9632
2026-08-12T06:54:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4742 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:27 dist-test-slave-6zbq krb5kdc[9864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:27 dist-test-slave-6zbq krb5kdc[9864](info): set up 2 sockets
Aug 12 06:54:27 dist-test-slave-6zbq krb5kdc[9864](info): commencing operation
krb5kdc: starting...
W20260812 06:54:28.853693 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:54:28 dist-test-slave-6zbq krb5kdc[9864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:54:29.074740 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37401
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:29.379961 9886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:29.380226 9886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:29.380363 9886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:29.387180 9886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:29.387286 9886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:29.387357 9886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:29.387432 9886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:29.387491 9886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:29.396518 9886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:29.397559 9886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:29.398739 9886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:29.409200 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:29.409263 9894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:29.409237 9896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:29.410140 9886 server_base.cc:1034] running on GCE node
I20260812 06:54:29.410840 9886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:29.412065 9886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:29.413259 9886 hybrid_clock.cc:648] HybridClock initialized: now 1786517669413221 us; error 28 us; skew 500 ppm
I20260812 06:54:29.413542 9886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:29 dist-test-slave-6zbq krb5kdc[9864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:29.419667 9886 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:29.420922 9886 webserver.cc:460] Webserver started at http://127.17.69.126:34225/ using document root <none> and password file <none>
I20260812 06:54:29.421517 9886 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:29.421615 9886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:29.421854 9886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:29.424506 9886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "ae05aa7f9d1c42128010c4215a063362"
format_stamp: "Formatted at 2026-08-12 06:54:29 on dist-test-slave-6zbq"
server_key: "b302a84236e1132a7d30aa9d14e08c02"
server_key_iv: "198c9b99d483d8bf8de6a2002ddad316"
server_key_version: "encryptionkey@0"
I20260812 06:54:29.425164 9886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "ae05aa7f9d1c42128010c4215a063362"
format_stamp: "Formatted at 2026-08-12 06:54:29 on dist-test-slave-6zbq"
server_key: "b302a84236e1132a7d30aa9d14e08c02"
server_key_iv: "198c9b99d483d8bf8de6a2002ddad316"
server_key_version: "encryptionkey@0"
I20260812 06:54:29.429683 9886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 06:54:29.432763 9905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:29.433866 9886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:54:29.433988 9886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "ae05aa7f9d1c42128010c4215a063362"
format_stamp: "Formatted at 2026-08-12 06:54:29 on dist-test-slave-6zbq"
server_key: "b302a84236e1132a7d30aa9d14e08c02"
server_key_iv: "198c9b99d483d8bf8de6a2002ddad316"
server_key_version: "encryptionkey@0"
I20260812 06:54:29.434134 9886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:29.445394 9886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:29.448871 9886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:29.449081 9886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:29.465324 9886 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:37401
I20260812 06:54:29.465342 9983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:37401 every 8 connection(s)
I20260812 06:54:29.466565 9886 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:29.469928 9984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:29.472119 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 9886
I20260812 06:54:29.472326 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:54:29.472955 17685 external_mini_cluster.cc:1428] Setting key 992882681ccb3900571a80b73ecaa628
I20260812 06:54:29.481855 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362: Bootstrap starting.
I20260812 06:54:29.485164 9984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:29.486085 9984 log.cc:824] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:54:29 dist-test-slave-6zbq krb5kdc[9864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:29.488829 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362: No bootstrap required, opened a new log
I20260812 06:54:29.496325 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:29.474649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.17.69.126:37401)
0812 06:54:29.475458 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:29.475471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:29.475518 (+ 47us) server_negotiation.cc:407] Connection header received
0812 06:54:29.477741 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:29.477780 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:29.478247 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:29.478686 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:29.479978 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:29.481896 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:29.483196 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:29.484188 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:29.484368 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:29.488284 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:29.488296 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:29.488313 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:29.488353 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:29.492126 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:29.492797 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:29.492809 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:29.492816 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:29.493037 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:29.493357 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:29.493362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:29.493365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:29.494042 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:29.494368 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:29.494684 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 06:54:29.495106 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":347,"threads_started":1}
I20260812 06:54:29.498662 9984 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } }
I20260812 06:54:29.499045 9984 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:29.499154 9984 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae05aa7f9d1c42128010c4215a063362, State: Initialized, Role: FOLLOWER
I20260812 06:54:29.499711 9984 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } }
I20260812 06:54:29.499941 9984 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:29.500077 9984 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:29.500114 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:54:29.500223 9984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:29.502240 9984 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } }
I20260812 06:54:29.502719 9984 leader_election.cc:302] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae05aa7f9d1c42128010c4215a063362; no voters:
I20260812 06:54:29.503278 9984 leader_election.cc:288] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:29.503347 9995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:29.503968 9995 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [term 1 LEADER]: Becoming Leader. State: Replica: ae05aa7f9d1c42128010c4215a063362, State: Running, Role: LEADER
I20260812 06:54:29.504431 9995 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } }
I20260812 06:54:29.505038 9984 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:29.507011 9997 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae05aa7f9d1c42128010c4215a063362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } } }
I20260812 06:54:29.507027 9999 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae05aa7f9d1c42128010c4215a063362. Latest consensus state: current_term: 1 leader_uuid: "ae05aa7f9d1c42128010c4215a063362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae05aa7f9d1c42128010c4215a063362" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37401 } } }
I20260812 06:54:29.507424 9997 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:29.507458 9999 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362 [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:29.508270 10003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:29.513497 10003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:29.521422 10003 catalog_manager.cc:1347] Generated new cluster ID: 5750d849c5f74b58a070107cbfe1a70c
I20260812 06:54:29.521510 10003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:29.544324 10003 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:29.545426 10003 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:29.552256 10003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae05aa7f9d1c42128010c4215a063362: Generated new TSK 0
I20260812 06:54:29.552883 10003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:54:29.559978 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 9886
I20260812 06:54:29.561002 9975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:29.647256 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 9886
2026-08-12T06:54:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2046 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:29 dist-test-slave-6zbq krb5kdc[10035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:29 dist-test-slave-6zbq krb5kdc[10035](info): set up 2 sockets
Aug 12 06:54:29 dist-test-slave-6zbq krb5kdc[10035](info): commencing operation
krb5kdc: starting...
W20260812 06:54:30.873306 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:54:31 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:54:31.093659 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:31.403910 10057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:31.404160 10057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:31.404282 10057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:31.411180 10057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:31.411273 10057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:31.411345 10057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:31.411414 10057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:31.411480 10057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:31.420614 10057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:31.421744 10057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:31.422909 10057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:31.433601 10066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:31.433652 10070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:31.433674 10067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:31.434623 10057 server_base.cc:1034] running on GCE node
I20260812 06:54:31.435328 10057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:31.436580 10057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:31.437785 10057 hybrid_clock.cc:648] HybridClock initialized: now 1786517671437735 us; error 39 us; skew 500 ppm
I20260812 06:54:31.438079 10057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:31 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:31.440987 10057 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:31.442438 10057 webserver.cc:460] Webserver started at http://127.17.69.126:39233/ using document root <none> and password file <none>
I20260812 06:54:31.443027 10057 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:31.443149 10057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:31.443421 10057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:31.446115 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "c7d103b9f60f49b582378d50651040ce"
format_stamp: "Formatted at 2026-08-12 06:54:31 on dist-test-slave-6zbq"
server_key: "24e3ba6ad5fd44ed00dbff1473851c80"
server_key_iv: "102133609765db32ede8955102d0823c"
server_key_version: "encryptionkey@0"
I20260812 06:54:31.446864 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "c7d103b9f60f49b582378d50651040ce"
format_stamp: "Formatted at 2026-08-12 06:54:31 on dist-test-slave-6zbq"
server_key: "24e3ba6ad5fd44ed00dbff1473851c80"
server_key_iv: "102133609765db32ede8955102d0823c"
server_key_version: "encryptionkey@0"
I20260812 06:54:31.451488 10057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:54:31.454614 10082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:31.455827 10057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:31.455996 10057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "c7d103b9f60f49b582378d50651040ce"
format_stamp: "Formatted at 2026-08-12 06:54:31 on dist-test-slave-6zbq"
server_key: "24e3ba6ad5fd44ed00dbff1473851c80"
server_key_iv: "102133609765db32ede8955102d0823c"
server_key_version: "encryptionkey@0"
I20260812 06:54:31.456179 10057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:31.496436 10057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:31.499938 10057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:31.500185 10057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:31.516916 10057 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40315
I20260812 06:54:31.516952 10162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40315 every 8 connection(s)
I20260812 06:54:31.518067 10057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:31.521371 10163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:31.521776 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 10057
I20260812 06:54:31.521968 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:54:31.522651 17685 external_mini_cluster.cc:1428] Setting key 0ec99040ffd76ec72af1d53e59af36aa
I20260812 06:54:31.534521 10163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap starting.
Aug 12 06:54:31 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:31.537364 10163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:31.538580 10163 log.cc:824] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:31.541759 10163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: No bootstrap required, opened a new log
I20260812 06:54:31.545668 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:31.524478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.17.69.126:40315)
0812 06:54:31.525246 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:31.525260 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:31.525317 (+ 57us) server_negotiation.cc:407] Connection header received
0812 06:54:31.527384 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:31.527410 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:31.527848 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:31.528268 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:31.529530 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:31.531503 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:31.532732 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:31.533715 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:31.533881 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:31.537793 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:31.537807 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:31.537828 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:31.537874 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:31.541563 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:31.542203 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:31.542211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:31.542218 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:31.542455 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:31.542796 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:31.542801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:31.542805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:31.543450 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:31.543752 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:31.544015 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 06:54:31.544438 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":300,"threads_started":1}
I20260812 06:54:31.550107 10163 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:31.550419 10163 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:31.550503 10163 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Initialized, Role: FOLLOWER
I20260812 06:54:31.550959 10163 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:31.551148 10163 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:31.551240 10163 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:31.551355 10163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:31.553491 10163 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:31.553886 10163 leader_election.cc:302] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d103b9f60f49b582378d50651040ce; no voters:
I20260812 06:54:31.554456 10163 leader_election.cc:288] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:31.554512 10170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:31.555101 10170 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 LEADER]: Becoming Leader. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Running, Role: LEADER
I20260812 06:54:31.555586 10170 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:31.556236 10163 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:31.558431 10171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:31.558900 10171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:31.559355 10173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d103b9f60f49b582378d50651040ce. Latest consensus state: current_term: 1 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:31.559722 10173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:31.561112 10180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:31.565999 10180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:31.574153 10180 catalog_manager.cc:1347] Generated new cluster ID: f7a68e44acbf416bbe26d8cb42514b2b
I20260812 06:54:31.574246 10180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:31.585813 10180 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:31.587476 10180 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:31.596598 10180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Generated new TSK 0
I20260812 06:54:31.597443 10180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:54:31.659896 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:31.984757 10208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:31.985001 10208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:31.985133 10208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:31.992028 10208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:31.992142 10208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:31.992282 10208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:32.001231 10208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:32.002247 10208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:32.003436 10208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:32.015245 10217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:32.015292 10221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:32.015343 10218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:32.016196 10208 server_base.cc:1034] running on GCE node
I20260812 06:54:32.016752 10208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:32.017542 10208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:32.018733 10208 hybrid_clock.cc:648] HybridClock initialized: now 1786517672018691 us; error 29 us; skew 500 ppm
I20260812 06:54:32.019006 10208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:32 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517672, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:32.021935 10208 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:32.023217 10208 webserver.cc:460] Webserver started at http://127.17.69.65:33665/ using document root <none> and password file <none>
I20260812 06:54:32.023823 10208 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:32.023937 10208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:32.024207 10208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:32.026928 10208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "7eb6acd20c51532782b5fcb3cb3ee9a7"
server_key_iv: "887a93e49a7b7a5740c42cfa6b8f4d05"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.027665 10208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "7eb6acd20c51532782b5fcb3cb3ee9a7"
server_key_iv: "887a93e49a7b7a5740c42cfa6b8f4d05"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.032302 10208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:32.035420 10230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.036543 10208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:32.036695 10208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "7eb6acd20c51532782b5fcb3cb3ee9a7"
server_key_iv: "887a93e49a7b7a5740c42cfa6b8f4d05"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.036868 10208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:32.063511 10208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:32.067421 10208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:32.067679 10208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:32.068516 10208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:32.069715 10208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:32.069792 10208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.069909 10208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:32.069962 10208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.099097 10208 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:33747
I20260812 06:54:32.099174 10431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:33747 every 8 connection(s)
I20260812 06:54:32.100186 10208 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:32.106981 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 10208
I20260812 06:54:32.107177 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:32.107788 17685 external_mini_cluster.cc:1428] Setting key 549c86f8267b790da89fd699e114c38d
Aug 12 06:54:32 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517672, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:32.118785 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:32.102313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:36377 (local address 127.17.69.126:40315)
0812 06:54:32.102727 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:32.102732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:32.103607 (+ 875us) server_negotiation.cc:407] Connection header received
0812 06:54:32.105310 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:32.105315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:32.105397 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:32.105489 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:32.106888 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.107637 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.109323 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.109786 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.109892 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:32.113526 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:32.113531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:32.113535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:32.113566 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:32.115977 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:32.116753 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:32.116757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:32.116760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:32.116878 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:32.117277 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:32.117280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:32.117282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:32.117527 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:32.117638 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:32.118375 (+ 737us) server_negotiation.cc:299] Negotiation successful
0812 06:54:32.118544 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":180,"threads_started":1}
I20260812 06:54:32.119820 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:32.102821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:36377)
0812 06:54:32.103471 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:32.103506 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:32.105060 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:32.105717 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:32.105724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:32.106181 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:32.106727 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:32.106745 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.107789 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.107793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:32.109148 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:32.109170 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.110005 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.110013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:32.110274 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:32.110886 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:32.110911 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:32.113325 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:32.116107 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:32.116115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:32.116128 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:32.116649 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:32.116969 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:32.116973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:32.116976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:32.117188 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:32.117644 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:32.117651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:32.117976 (+ 325us) client_negotiation.cc:769] Sending connection context
0812 06:54:32.118230 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 06:54:32.118686 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":256,"threads_started":1}
I20260812 06:54:32.121140 10434 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:32.121416 10434 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:32.122098 10434 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:32.124192 10107 ts_manager.cc:194] Registered new tserver with Master: 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747)
I20260812 06:54:32.126034 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:36377
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:54:32.163164 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:32.486040 10442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:32.486275 10442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:32.486416 10442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:32.493289 10442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:32.493376 10442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:32.493484 10442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:32.502521 10442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:32.503515 10442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:32.504787 10442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:32.516757 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:32.516849 10452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:32.517179 10450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:32.518095 10442 server_base.cc:1034] running on GCE node
I20260812 06:54:32.518924 10442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:32.520036 10442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:32.521286 10442 hybrid_clock.cc:648] HybridClock initialized: now 1786517672521199 us; error 67 us; skew 500 ppm
I20260812 06:54:32.521613 10442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:32 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517672, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:32.525224 10442 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:32.526737 10442 webserver.cc:460] Webserver started at http://127.17.69.66:41089/ using document root <none> and password file <none>
I20260812 06:54:32.527457 10442 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:32.527572 10442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:32.527869 10442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:32.531471 10442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "33533b18acf242b28778b277e14f1ebd"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "b6a331e30c66303297663d29307ec710"
server_key_iv: "7f1f845a5509726f682727f6a1a65677"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.532423 10442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "33533b18acf242b28778b277e14f1ebd"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "b6a331e30c66303297663d29307ec710"
server_key_iv: "7f1f845a5509726f682727f6a1a65677"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.537873 10442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 06:54:32.541013 10468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.542292 10442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:32.542479 10442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "33533b18acf242b28778b277e14f1ebd"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "b6a331e30c66303297663d29307ec710"
server_key_iv: "7f1f845a5509726f682727f6a1a65677"
server_key_version: "encryptionkey@0"
I20260812 06:54:32.542634 10442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:32.556629 10442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:32.560330 10442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:32.560568 10442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:32.561441 10442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:32.562700 10442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:32.562770 10442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.562878 10442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:32.562929 10442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:32.591909 10442 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:34601
I20260812 06:54:32.591964 10664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:34601 every 8 connection(s)
I20260812 06:54:32.593010 10442 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:32.602633 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 10442
I20260812 06:54:32.602824 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:32.603272 17685 external_mini_cluster.cc:1428] Setting key 9c891bc9264c1a18bd4c17031a54ed3a
Aug 12 06:54:32 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517672, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:32.613060 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:32.595254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:45635 (local address 127.17.69.126:40315)
0812 06:54:32.595428 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:32.595432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:32.596430 (+ 998us) server_negotiation.cc:407] Connection header received
0812 06:54:32.598277 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:32.598283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:32.598392 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:32.598476 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:32.599866 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.600559 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.602115 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.602575 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.602694 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:32.607002 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:32.607009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:32.607014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:32.607050 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:32.610315 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:32.610971 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:32.610976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:32.610980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:32.611141 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:32.611506 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:32.611511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:32.611514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:32.611825 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:32.611955 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:32.612527 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 06:54:32.612748 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:54:32.613735 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:32.595588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:45635)
0812 06:54:32.596280 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:32.596322 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:32.598008 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:32.598717 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:32.598725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:32.599193 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:32.599708 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:32.599724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.600717 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.600721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:32.601981 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:32.601994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:32.602806 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:32.602813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:32.603040 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:32.604313 (+ 1273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:32.604344 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:32.606764 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:32.610456 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:32.610463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:32.610473 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:32.610873 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:32.611238 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:32.611241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:32.611243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:32.611413 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:32.611928 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:32.611933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:32.612212 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 06:54:32.612420 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 06:54:32.612765 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":256,"threads_started":1}
I20260812 06:54:32.615054 10665 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:32.615334 10665 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:32.615991 10665 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:32.617353 10107 ts_manager.cc:194] Registered new tserver with Master: 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
I20260812 06:54:32.618265 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:45635
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:54:32.657616 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:32.967134 10672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:32.967397 10672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:32.967530 10672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:32.974315 10672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:32.974434 10672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:32.974579 10672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:32.983528 10672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:32.984618 10672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:32.985817 10672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:32.998056 10681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:32.997988 10684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:32.998178 10672 server_base.cc:1034] running on GCE node
W20260812 06:54:32.998030 10687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:32.999119 10672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:32.999920 10672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:33.001121 10672 hybrid_clock.cc:648] HybridClock initialized: now 1786517673001077 us; error 35 us; skew 500 ppm
I20260812 06:54:33.001396 10672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:33 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:33.004252 10672 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:33.005539 10672 webserver.cc:460] Webserver started at http://127.17.69.67:34685/ using document root <none> and password file <none>
I20260812 06:54:33.006131 10672 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:33.006258 10672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:33.006568 10672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:33.009241 10672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
format_stamp: "Formatted at 2026-08-12 06:54:33 on dist-test-slave-6zbq"
server_key: "91355df24a38d176b45ef0d089b58b72"
server_key_iv: "2a9aebedd236b93133905255e74b5365"
server_key_version: "encryptionkey@0"
I20260812 06:54:33.009964 10672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
format_stamp: "Formatted at 2026-08-12 06:54:33 on dist-test-slave-6zbq"
server_key: "91355df24a38d176b45ef0d089b58b72"
server_key_iv: "2a9aebedd236b93133905255e74b5365"
server_key_version: "encryptionkey@0"
I20260812 06:54:33.014640 10672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260812 06:54:33.017741 10697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:33.018913 10672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:33.019049 10672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
format_stamp: "Formatted at 2026-08-12 06:54:33 on dist-test-slave-6zbq"
server_key: "91355df24a38d176b45ef0d089b58b72"
server_key_iv: "2a9aebedd236b93133905255e74b5365"
server_key_version: "encryptionkey@0"
I20260812 06:54:33.019219 10672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:33.036171 10672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:33.039836 10672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:33.040076 10672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:33.041034 10672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:33.042317 10672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:33.042424 10672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:33.042546 10672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:33.042608 10672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:33.072510 10672 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:37875
I20260812 06:54:33.072559 10874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:37875 every 8 connection(s)
I20260812 06:54:33.073639 10672 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:33.078586 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 10672
I20260812 06:54:33.078789 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:33.079344 17685 external_mini_cluster.cc:1428] Setting key bb1f77d86012fb5c9e74dafaa39fa158
Aug 12 06:54:33 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:33.092651 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.076001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:47613 (local address 127.17.69.126:40315)
0812 06:54:33.076185 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.076190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.077149 (+ 959us) server_negotiation.cc:407] Connection header received
0812 06:54:33.079172 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.079177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.079276 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.079360 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:33.081223 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.082039 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.083763 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.084180 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.084295 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.087509 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:33.087514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:33.087518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:33.087548 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:33.089839 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.090519 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.090524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.090526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.090660 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.091052 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.091056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.091058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.091316 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:33.091448 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.092082 (+ 634us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.092303 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:54:33.093452 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.076329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:47613)
0812 06:54:33.077001 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.077038 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.078882 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.079635 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.079646 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.080283 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:33.081014 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.081040 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.082295 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.082302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.083612 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.083625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.084401 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.084408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.084627 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.085215 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:33.085241 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:33.087316 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:33.089962 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:33.089970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:33.089985 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:33.090405 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:33.090761 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:33.090766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:33.090769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:33.090942 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:33.091433 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:33.091440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:33.091735 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.091954 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.092421 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
I20260812 06:54:33.094772 10875 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:33.095055 10875 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:33.095679 10875 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:33.096951 10107 ts_manager.cc:194] Registered new tserver with Master: 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875)
I20260812 06:54:33.097728 10107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:47613
I20260812 06:54:33.106598 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:33.121805 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.110829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.17.69.126:40315)
0812 06:54:33.111016 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.111023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.111318 (+ 295us) server_negotiation.cc:407] Connection header received
0812 06:54:33.111455 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.111459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.111514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.111579 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:33.112917 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.113601 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.114884 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.115285 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.115376 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.117392 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:33.117397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:33.117400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:33.117429 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:33.119643 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.120249 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.120252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.120255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.120369 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.120660 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.120663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.120664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.120952 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:33.121139 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.121389 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.121534 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:54:33.128660 10434 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:33.125267 10107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:33.132145 10107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:33.159085 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.143557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37875 (local address 127.0.0.1:59278)
0812 06:54:33.144430 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.144448 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.144550 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.152498 (+ 7948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.152502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.152526 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:33.152971 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.152982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.154318 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.154323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.156621 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.156636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.158067 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.158074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.158270 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.158307 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.158462 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.158635 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":25,"thread_start_us":7361,"threads_started":1}
I20260812 06:54:33.159349 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.143149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.17.69.65:33747)
0812 06:54:33.150828 (+ 7679us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.150836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.150865 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:54:33.150957 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.150965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.152001 (+ 1036us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.152774 (+ 773us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:33.153001 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.154563 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.156956 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.158189 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.158388 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.158603 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.158811 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.158964 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7525,"thread_start_us":7378,"threads_started":1}
I20260812 06:54:33.159265 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.142862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:36102)
0812 06:54:33.143622 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.143665 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.143849 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.152251 (+ 8402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.152259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.152288 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:33.152669 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.152685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.154696 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.154702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.156803 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.156816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.158470 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.158477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.158629 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.158659 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.158766 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.158884 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":1,"thread_start_us":351,"threads_started":1}
I20260812 06:54:33.159770 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.143940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34601 (local address 127.0.0.1:44796)
0812 06:54:33.151165 (+ 7225us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.151181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.151292 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.152858 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.152862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.152890 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:33.153313 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.153326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.155189 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.155195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.157151 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.157165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.158951 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.158958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.159183 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.159221 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.159295 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.159411 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7119,"mutex_wait_us":9,"thread_start_us":637,"threads_started":1}
I20260812 06:54:33.159904 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.144124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.17.69.67:37875)
0812 06:54:33.151832 (+ 7708us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.151839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.151868 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:54:33.151954 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.151962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.152177 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.152476 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:33.153149 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.154196 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.156862 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.157888 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.158002 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.158224 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.159403 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.159564 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7536,"thread_start_us":7395,"threads_started":1}
I20260812 06:54:33.160998 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.144404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.17.69.66:34601)
0812 06:54:33.152302 (+ 7898us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.152310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.152339 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:54:33.152440 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.152446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.152636 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.153500 (+ 864us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:33.153702 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.155000 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.157323 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.158708 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.160031 (+ 1323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.160332 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.160461 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.160618 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7744,"thread_start_us":7624,"threads_started":1}
I20260812 06:54:33.163079 10318 tablet_service.cc:1467] Processing CreateTablet for tablet f770c4144ae54518a6cfcb7d69519257 (DEFAULT_TABLE table=test-table [id=a7ada1542c3e44889d211ff2d8ef5952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:33.163486 10767 tablet_service.cc:1467] Processing CreateTablet for tablet f770c4144ae54518a6cfcb7d69519257 (DEFAULT_TABLE table=test-table [id=a7ada1542c3e44889d211ff2d8ef5952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:33.164250 10318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f770c4144ae54518a6cfcb7d69519257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:33.164562 10767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f770c4144ae54518a6cfcb7d69519257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:33.169312 10550 tablet_service.cc:1467] Processing CreateTablet for tablet f770c4144ae54518a6cfcb7d69519257 (DEFAULT_TABLE table=test-table [id=a7ada1542c3e44889d211ff2d8ef5952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:33.170603 10550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f770c4144ae54518a6cfcb7d69519257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:33.175343 10903 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Bootstrap starting.
I20260812 06:54:33.177220 10904 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Bootstrap starting.
I20260812 06:54:33.178833 10903 tablet_bootstrap.cc:654] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:33.180033 10903 log.cc:824] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:33.180423 10904 tablet_bootstrap.cc:654] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:33.181461 10904 log.cc:824] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:33.183121 10903 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: No bootstrap required, opened a new log
I20260812 06:54:33.183372 10903 ts_tablet_manager.cc:1397] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260812 06:54:33.184413 10904 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: No bootstrap required, opened a new log
I20260812 06:54:33.184675 10904 ts_tablet_manager.cc:1397] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260812 06:54:33.187052 10911 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Bootstrap starting.
I20260812 06:54:33.190452 10911 tablet_bootstrap.cc:654] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:33.191593 10911 log.cc:824] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:33.191727 10903 raft_consensus.cc:348] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.192203 10903 raft_consensus.cc:374] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:33.192328 10903 raft_consensus.cc:729] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dc17dedc1c4aa2920090c0ad9f954f, State: Initialized, Role: FOLLOWER
I20260812 06:54:33.192185 10904 raft_consensus.cc:348] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.192572 10904 raft_consensus.cc:374] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:33.192667 10904 raft_consensus.cc:729] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dfbd42eb8b445d8f6d84d45c7ed420, State: Initialized, Role: FOLLOWER
I20260812 06:54:33.193019 10903 consensus_queue.cc:260] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.193199 10904 consensus_queue.cc:260] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.194631 10875 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:33.195868 10904 ts_tablet_manager.cc:1428] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260812 06:54:33.196447 10903 ts_tablet_manager.cc:1428] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260812 06:54:33.196732 10911 tablet_bootstrap.cc:492] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: No bootstrap required, opened a new log
I20260812 06:54:33.196951 10911 ts_tablet_manager.cc:1397] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 06:54:33.203909 10911 raft_consensus.cc:348] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.204208 10911 raft_consensus.cc:374] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:33.204293 10911 raft_consensus.cc:729] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33533b18acf242b28778b277e14f1ebd, State: Initialized, Role: FOLLOWER
I20260812 06:54:33.204711 10911 consensus_queue.cc:260] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.205590 10665 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:33.206156 10911 ts_tablet_manager.cc:1428] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260812 06:54:33.262859 10916 raft_consensus.cc:482] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:33.263159 10916 raft_consensus.cc:504] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.265138 10916 leader_election.cc:288] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601), 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875)
I20260812 06:54:33.271595 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.265634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:51403 (local address 127.17.69.66:34601)
0812 06:54:33.265859 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.265865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.266501 (+ 636us) server_negotiation.cc:407] Connection header received
0812 06:54:33.266669 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.266674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.266731 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.266808 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:33.267422 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.268099 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.269886 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.270639 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.270782 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.270858 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.271212 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.271346 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:54:33.271584 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.265645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:53377 (local address 127.17.69.67:37875)
0812 06:54:33.265822 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.265828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.266288 (+ 460us) server_negotiation.cc:407] Connection header received
0812 06:54:33.266518 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.266522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.266583 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.266664 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:33.267306 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.268039 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.269840 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.270641 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.270789 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.270860 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.271209 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.271336 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 06:54:33.271747 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.265381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37875 (local address 127.17.69.65:53377)
0812 06:54:33.266173 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.266189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.266363 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.266699 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.266703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.266728 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:33.267155 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.267166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.268182 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.268188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.269689 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.269702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.270866 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.270874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.271019 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.271072 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.271162 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.271336 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":47,"spinlock_wait_cycles":33152,"thread_start_us":453,"threads_started":1}
I20260812 06:54:33.271914 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.265391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34601 (local address 127.17.69.65:51403)
0812 06:54:33.266396 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:33.266416 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:33.266541 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:33.266848 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:33.266853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:33.266905 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:33.267280 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.267289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.268239 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.268243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.269729 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:33.269743 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.270917 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.270922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:33.271066 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.271088 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:54:33.271166 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:54:33.271331 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":253,"spinlock_wait_cycles":768,"thread_start_us":462,"threads_started":1}
I20260812 06:54:33.272688 10801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f770c4144ae54518a6cfcb7d69519257" candidate_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" is_pre_election: true
I20260812 06:54:33.272782 10581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f770c4144ae54518a6cfcb7d69519257" candidate_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33533b18acf242b28778b277e14f1ebd" is_pre_election: true
I20260812 06:54:33.273276 10581 raft_consensus.cc:2454] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92dfbd42eb8b445d8f6d84d45c7ed420 in term 0.
I20260812 06:54:33.273276 10801 raft_consensus.cc:2454] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92dfbd42eb8b445d8f6d84d45c7ed420 in term 0.
I20260812 06:54:33.273912 10232 leader_election.cc:302] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33533b18acf242b28778b277e14f1ebd, 92dfbd42eb8b445d8f6d84d45c7ed420; no voters:
I20260812 06:54:33.274237 10916 raft_consensus.cc:2781] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:33.274406 10916 raft_consensus.cc:482] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:33.274518 10916 raft_consensus.cc:3037] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:33.276644 10916 raft_consensus.cc:504] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.277227 10916 leader_election.cc:288] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 1 election: Requested vote from peers 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601), 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875)
I20260812 06:54:33.277647 10581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f770c4144ae54518a6cfcb7d69519257" candidate_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33533b18acf242b28778b277e14f1ebd"
I20260812 06:54:33.277704 10801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f770c4144ae54518a6cfcb7d69519257" candidate_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
I20260812 06:54:33.277887 10581 raft_consensus.cc:3037] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:33.277967 10801 raft_consensus.cc:3037] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:33.280159 10801 raft_consensus.cc:2454] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92dfbd42eb8b445d8f6d84d45c7ed420 in term 1.
I20260812 06:54:33.280192 10581 raft_consensus.cc:2454] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92dfbd42eb8b445d8f6d84d45c7ed420 in term 1.
I20260812 06:54:33.280563 10233 leader_election.cc:302] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f, 92dfbd42eb8b445d8f6d84d45c7ed420; no voters:
I20260812 06:54:33.280836 10916 raft_consensus.cc:2781] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:33.281270 10916 raft_consensus.cc:686] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 LEADER]: Becoming Leader. State: Replica: 92dfbd42eb8b445d8f6d84d45c7ed420, State: Running, Role: LEADER
I20260812 06:54:33.281766 10916 consensus_queue.cc:237] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } }
I20260812 06:54:33.285974 10106 catalog_manager.cc:5495] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65). New cstate: current_term: 1 leader_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } health_report { overall_health: UNKNOWN } } }
I20260812 06:54:33.305876 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.300919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36106 (local address 127.17.69.65:33747)
0812 06:54:33.301087 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.301092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.301109 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:54:33.301283 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.301286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.301362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.301471 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:33.301760 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.302473 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.304031 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.304455 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.304555 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.304982 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:33.305426 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:33.305526 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.305630 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.305695 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:54:33.313196 10801 raft_consensus.cc:1261] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Refusing update from remote peer 92dfbd42eb8b445d8f6d84d45c7ed420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:33.313202 10581 raft_consensus.cc:1261] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Refusing update from remote peer 92dfbd42eb8b445d8f6d84d45c7ed420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:33.314059 10920 consensus_queue.cc:1035] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:33.314407 10916 consensus_queue.cc:1035] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:33.342097 10107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:33.342410 10107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:33.346107 10107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Sending DeleteTablet for 3 replicas of tablet f770c4144ae54518a6cfcb7d69519257
I20260812 06:54:33.347155 10318 tablet_service.cc:1514] Processing DeleteTablet for tablet f770c4144ae54518a6cfcb7d69519257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:33 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:36102
I20260812 06:54:33.347254 10550 tablet_service.cc:1514] Processing DeleteTablet for tablet f770c4144ae54518a6cfcb7d69519257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:33 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:44796
I20260812 06:54:33.347254 10767 tablet_service.cc:1514] Processing DeleteTablet for tablet f770c4144ae54518a6cfcb7d69519257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:33 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:59278
I20260812 06:54:33.349520 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 10208
I20260812 06:54:33.357146 10945 tablet_replica.cc:330] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: stopping tablet replica
I20260812 06:54:33.357452 10944 tablet_replica.cc:330] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: stopping tablet replica
I20260812 06:54:33.357851 10944 raft_consensus.cc:2229] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:33.358544 10944 raft_consensus.cc:2258] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:33.359722 10948 tablet_replica.cc:330] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: stopping tablet replica
I20260812 06:54:33.360143 10948 raft_consensus.cc:2229] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:33.360540 10944 ts_tablet_manager.cc:1905] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:33.360656 10948 raft_consensus.cc:2258] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:33.358788 10945 raft_consensus.cc:2229] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:33.361678 10945 raft_consensus.cc:2258] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:33.363349 10948 ts_tablet_manager.cc:1905] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:33.367753 10945 ts_tablet_manager.cc:1905] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:33.367830 10944 ts_tablet_manager.cc:1918] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:33.367998 10944 log.cc:1196] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/f770c4144ae54518a6cfcb7d69519257
I20260812 06:54:33.368558 10944 ts_tablet_manager.cc:1939] T f770c4144ae54518a6cfcb7d69519257 P 92dfbd42eb8b445d8f6d84d45c7ed420: Deleting consensus metadata
Aug 12 06:54:33 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:54:33.370141 10085 catalog_manager.cc:4841] TS 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747): tablet f770c4144ae54518a6cfcb7d69519257 (table test-table [id=a7ada1542c3e44889d211ff2d8ef5952]) successfully deleted
I20260812 06:54:33.374141 10948 ts_tablet_manager.cc:1918] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:33.374297 10948 log.cc:1196] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/f770c4144ae54518a6cfcb7d69519257
I20260812 06:54:33.374818 10948 ts_tablet_manager.cc:1939] T f770c4144ae54518a6cfcb7d69519257 P 33533b18acf242b28778b277e14f1ebd: Deleting consensus metadata
I20260812 06:54:33.376463 10085 catalog_manager.cc:4841] TS 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): tablet f770c4144ae54518a6cfcb7d69519257 (table test-table [id=a7ada1542c3e44889d211ff2d8ef5952]) successfully deleted
I20260812 06:54:33.378309 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.350802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.17.69.65:33747)
0812 06:54:33.350973 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.350979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.357789 (+ 6810us) server_negotiation.cc:407] Connection header received
0812 06:54:33.357999 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.358005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.358101 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.358210 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:33.360555 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.361727 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.364268 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.365058 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.365181 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.370982 (+ 5801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:33.370996 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:33.371016 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:33.371072 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:33.375004 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.375932 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.375941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.375945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.376123 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.376625 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.376630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.376633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.377013 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:33.377217 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.377723 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.377948 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:54:33.378948 10424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:33.379362 10945 ts_tablet_manager.cc:1918] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:33.379590 10945 log.cc:1196] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/f770c4144ae54518a6cfcb7d69519257
I20260812 06:54:33.380139 10945 ts_tablet_manager.cc:1939] T f770c4144ae54518a6cfcb7d69519257 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting consensus metadata
I20260812 06:54:33.381714 10086 catalog_manager.cc:4841] TS 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): tablet f770c4144ae54518a6cfcb7d69519257 (table test-table [id=a7ada1542c3e44889d211ff2d8ef5952]) successfully deleted
I20260812 06:54:33.476368 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 10208
I20260812 06:54:33.500283 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 10442
Aug 12 06:54:33 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:54:33.514727 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.501009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.17.69.66:34601)
0812 06:54:33.501222 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.501227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.501245 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:33.501412 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.501416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.501479 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.501560 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:33.502799 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.503546 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.504733 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.505112 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.505227 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.509370 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:33.509385 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:33.509400 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:33.509436 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:33.512053 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.512693 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.512702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.512704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.512844 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.513189 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.513194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.513197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.513570 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:33.513843 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.514168 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.514395 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 06:54:33.515429 10656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:33.602629 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 10442
I20260812 06:54:33.626056 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 10672
Aug 12 06:54:33 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:54:33.640610 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:33.626831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.17.69.67:37875)
0812 06:54:33.627015 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:33.627021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:33.627081 (+ 60us) server_negotiation.cc:407] Connection header received
0812 06:54:33.627286 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:33.627292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:33.627364 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:33.627450 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:33.628767 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.629487 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.630766 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:33.631219 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:33.631335 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:33.635532 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:33.635547 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:33.635563 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:33.635600 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:33.637987 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.638648 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.638662 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.638664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.638803 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:33.639151 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:33.639156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:33.639159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:33.639509 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:33.639793 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:33.640108 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 06:54:33.640322 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:54:33.641247 10868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:33.727739 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 10672
I20260812 06:54:33.747469 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 10057
I20260812 06:54:33.748540 10154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:33.858433 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 10057
I20260812 06:54:33.884438 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--webserver_interface=127.17.69.126
--webserver_port=39233
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:34.202329 10961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:34.202626 10961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:34.202757 10961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:34.209697 10961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:34.209784 10961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:34.209854 10961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:34.209923 10961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:34.209980 10961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:34.219224 10961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=39233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:34.220362 10961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:34.221556 10961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:34.244263 10969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:34.244287 10972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:34.244266 10976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:34.244894 10961 server_base.cc:1034] running on GCE node
I20260812 06:54:34.245657 10961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:34.247159 10961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:34.248363 10961 hybrid_clock.cc:648] HybridClock initialized: now 1786517674248312 us; error 42 us; skew 500 ppm
I20260812 06:54:34.248730 10961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:34 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:34.252031 10961 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:34.253536 10961 webserver.cc:460] Webserver started at http://127.17.69.126:39233/ using document root <none> and password file <none>
I20260812 06:54:34.254269 10961 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:34.254419 10961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:34.259895 10961 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:54:34.262600 10986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:34.263799 10961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:34.263960 10961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "c7d103b9f60f49b582378d50651040ce"
format_stamp: "Formatted at 2026-08-12 06:54:31 on dist-test-slave-6zbq"
server_key: "24e3ba6ad5fd44ed00dbff1473851c80"
server_key_iv: "102133609765db32ede8955102d0823c"
server_key_version: "encryptionkey@0"
I20260812 06:54:34.264968 10961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:34.297590 10961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:34.301589 10961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:34.301836 10961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:34.319599 10961 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40315
I20260812 06:54:34.319617 11062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40315 every 8 connection(s)
I20260812 06:54:34.321029 10961 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:34.327627 11063 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:54:34.328083 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 10961
I20260812 06:54:34.329028 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:33747
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=33665
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:34.332005 11063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap starting.
I20260812 06:54:34.352788 11063 log.cc:824] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:34.364826 11063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:54:34.365335 11063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap complete.
I20260812 06:54:34.374434 11063 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:34.375218 11063 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Initialized, Role: FOLLOWER
I20260812 06:54:34.375780 11063 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:34.376003 11063 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:34.376114 11063 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:34.376269 11063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:54:34.379943 11063 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:34.380429 11063 leader_election.cc:302] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d103b9f60f49b582378d50651040ce; no voters:
I20260812 06:54:34.381011 11063 leader_election.cc:288] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:54:34.381217 11073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:54:34.382365 11073 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 LEADER]: Becoming Leader. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Running, Role: LEADER
I20260812 06:54:34.382893 11073 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:34.383255 11063 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:34.385471 11078 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d103b9f60f49b582378d50651040ce. Latest consensus state: current_term: 2 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:34.385715 11074 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:34.386052 11074 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:34.385946 11078 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:34.399998 11089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:34.408241 11089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:34.409638 11089 catalog_manager.cc:1259] Loaded cluster ID: f7a68e44acbf416bbe26d8cb42514b2b
I20260812 06:54:34.409794 11089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:34.412956 11089 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 06:54:34.414944 11107 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:54:34.430002 11089 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 06:54:34.438798 11089 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:34.440429 11089 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Loaded TSK: 0
I20260812 06:54:34.441913 11089 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:54:34.675366 11069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:34.675637 11069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:34.675771 11069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:34.682785 11069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:34.682894 11069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:34.683050 11069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:34.692310 11069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:33747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=33665
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:34.693460 11069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:34.694761 11069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:34.708575 11116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:34.708977 11121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:34.709460 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:34.710094 11069 server_base.cc:1034] running on GCE node
I20260812 06:54:34.710863 11069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:34.711839 11069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:34.713063 11069 hybrid_clock.cc:648] HybridClock initialized: now 1786517674713006 us; error 45 us; skew 500 ppm
I20260812 06:54:34.713378 11069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:34 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:34.720564 11069 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:34.722190 11069 webserver.cc:460] Webserver started at http://127.17.69.65:33665/ using document root <none> and password file <none>
I20260812 06:54:34.723034 11069 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:34.723188 11069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:34.729925 11069 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:54:34.733244 11137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:34.734700 11069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 06:54:34.734871 11069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "7eb6acd20c51532782b5fcb3cb3ee9a7"
server_key_iv: "887a93e49a7b7a5740c42cfa6b8f4d05"
server_key_version: "encryptionkey@0"
I20260812 06:54:34.736181 11069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:34.761914 11069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:34.765786 11069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:34.766007 11069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:34.766961 11069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:34.768219 11069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:34.768317 11069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:34.768433 11069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:34.768514 11069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:34.798946 11069 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:33747
I20260812 06:54:34.799015 11333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:33747 every 8 connection(s)
I20260812 06:54:34.800493 11069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:34.807894 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 11069
I20260812 06:54:34.808530 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:34601
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=41089
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:34 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:34.822410 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:34.803446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:53803)
0812 06:54:34.804229 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:34.804273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:34.806050 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:34.807407 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:34.807415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:34.807935 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:34.808523 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:34.808541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:34.810251 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:34.810255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:34.811483 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:34.811496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:34.812494 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:34.812503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:34.812745 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:34.813370 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:34.813397 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:34.815439 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:34.818454 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:34.818462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:34.818476 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:34.818861 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:34.819267 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:34.819272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:34.819275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:34.819467 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:34.820142 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:34.820149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:34.820451 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 06:54:34.820673 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 06:54:34.821175 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":322,"threads_started":1}
I20260812 06:54:34.822402 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:34.803291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:53803 (local address 127.17.69.126:40315)
0812 06:54:34.803839 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:34.803851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:34.804388 (+ 537us) server_negotiation.cc:407] Connection header received
0812 06:54:34.806589 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:34.806603 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:34.806928 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:34.807231 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:34.808756 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:34.810087 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:34.811621 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:34.812261 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:34.812438 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:34.815688 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:34.815695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:34.815708 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:34.815742 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:34.818297 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:34.818995 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:34.819001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:34.819007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:34.819160 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:34.819570 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:34.819574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:34.819576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:34.820011 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:34.820193 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:34.820893 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 06:54:34.821290 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":225,"threads_started":1}
I20260812 06:54:34.825301 11337 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:34.825605 11337 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:34.826458 11337 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:34.828980 11009 ts_manager.cc:194] Registered new tserver with Master: 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747)
I20260812 06:54:34.831022 11009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:53803
W20260812 06:54:35.131640 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:35.131884 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:35.131981 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:35.138800 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:35.138888 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:35.138993 11344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:35.148116 11344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:34601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=41089
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:35.149272 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:35.150489 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:35.162493 11350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:35.162529 11351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:35.162567 11356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:35.163722 11344 server_base.cc:1034] running on GCE node
I20260812 06:54:35.164320 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:35.165194 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:35.166409 11344 hybrid_clock.cc:648] HybridClock initialized: now 1786517675166361 us; error 36 us; skew 500 ppm
I20260812 06:54:35.166702 11344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:35 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:35.169736 11344 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:35.171038 11344 webserver.cc:460] Webserver started at http://127.17.69.66:41089/ using document root <none> and password file <none>
I20260812 06:54:35.171671 11344 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:35.171797 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:35.176656 11344 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:54:35.179127 11367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.180454 11344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:35.180629 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "33533b18acf242b28778b277e14f1ebd"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "b6a331e30c66303297663d29307ec710"
server_key_iv: "7f1f845a5509726f682727f6a1a65677"
server_key_version: "encryptionkey@0"
I20260812 06:54:35.181558 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:35.211946 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:35.215564 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:35.215812 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:35.216694 11344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:35.217923 11344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:35.218016 11344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.218119 11344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:35.218204 11344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.247340 11344 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:34601
I20260812 06:54:35.247404 11561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:34601 every 8 connection(s)
I20260812 06:54:35.248580 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:35.256628 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 11344
I20260812 06:54:35.257196 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:37875
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=34685
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:54:35 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:35.267169 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.250791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:44527 (local address 127.17.69.126:40315)
0812 06:54:35.250950 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.250954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.252021 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 06:54:35.253755 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.253762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.253842 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.253927 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:35.255424 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.256174 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.257755 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.258164 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.258273 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.261540 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:35.261546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:35.261550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:35.261592 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:35.264430 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.265063 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.265068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.265072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.265219 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.265577 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.265581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.265584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.265906 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:35.266045 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.266661 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.266871 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:54:35.267827 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.251197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:44527)
0812 06:54:35.251862 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.251898 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.253495 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.254170 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.254178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.254722 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:35.255254 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.255272 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.256322 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.256326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.257607 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.257622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.258401 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.258410 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.258611 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.259204 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:35.259224 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:35.261333 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:35.264551 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:35.264557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:35.264567 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:35.264954 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:35.265310 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:35.265313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:35.265316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:35.265483 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:35.266005 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:35.266010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:35.266287 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.266532 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.266873 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":265,"threads_started":1}
I20260812 06:54:35.269250 11562 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:35.269508 11562 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:35.270166 11562 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:35.271528 11009 ts_manager.cc:194] Registered new tserver with Master: 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
I20260812 06:54:35.272310 11009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:44527
W20260812 06:54:35.580161 11573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:35.580420 11573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:35.580555 11573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:35.587637 11573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:35.587749 11573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:35.587893 11573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:35.597162 11573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:37875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=34685
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:35.598263 11573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:35.599534 11573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:35.611661 11583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:35.611608 11587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:35.611702 11590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:35.612349 11573 server_base.cc:1034] running on GCE node
I20260812 06:54:35.612884 11573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:35.613734 11573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:35.614920 11573 hybrid_clock.cc:648] HybridClock initialized: now 1786517675614868 us; error 42 us; skew 500 ppm
I20260812 06:54:35.615199 11573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:35 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:35.618084 11573 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:35.619413 11573 webserver.cc:460] Webserver started at http://127.17.69.67:34685/ using document root <none> and password file <none>
I20260812 06:54:35.620018 11573 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:35.620147 11573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:35.625053 11573 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:54:35.627588 11605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.628770 11573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:35.628906 11573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
format_stamp: "Formatted at 2026-08-12 06:54:33 on dist-test-slave-6zbq"
server_key: "91355df24a38d176b45ef0d089b58b72"
server_key_iv: "2a9aebedd236b93133905255e74b5365"
server_key_version: "encryptionkey@0"
I20260812 06:54:35.629822 11573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:35.657975 11573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:35.661983 11573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:35.662225 11573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:35.663153 11573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:35.664415 11573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:35.664508 11573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.664609 11573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:35.664678 11573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:35.694455 11573 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:37875
I20260812 06:54:35.694548 11815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:37875 every 8 connection(s)
I20260812 06:54:35.695792 11573 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:35.704753 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 11573
Aug 12 06:54:35 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:35.715302 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.698150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:42111 (local address 127.17.69.126:40315)
0812 06:54:35.698387 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.698394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.699460 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 06:54:35.701158 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.701162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.701230 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.701313 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:35.702669 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.703383 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.704886 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.705448 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.705565 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.709527 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:35.709532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:35.709536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:35.709565 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:35.712520 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.713212 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.713218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.713222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.713367 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.713741 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.713746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.713750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.714103 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:35.714252 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.714876 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.715073 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 06:54:35.716408 11822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.698662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:42111)
0812 06:54:35.699312 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.699354 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.700912 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.701484 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.701490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.701965 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:35.702527 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.702545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.703511 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.703514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.704714 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.704737 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.705686 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.705693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.705973 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.706744 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:35.706774 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:35.709328 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:35.712647 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:35.712656 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:35.712670 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:35.713073 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:35.713462 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:35.713465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:35.713468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:35.713630 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:35.714215 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:35.714221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:35.714518 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.714729 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.715184 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":250,"threads_started":1}
I20260812 06:54:35.717825 11818 heartbeater.cc:347] Connected to a master server at 127.17.69.126:40315
I20260812 06:54:35.718108 11818 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:35.718817 11818 heartbeater.cc:511] Master 127.17.69.126:40315 requested a full tablet report, sending...
I20260812 06:54:35.720263 11009 ts_manager.cc:194] Registered new tserver with Master: 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875)
I20260812 06:54:35.720557 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.705740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.17.69.126:40315)
0812 06:54:35.706224 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.706230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.706625 (+ 395us) server_negotiation.cc:407] Connection header received
0812 06:54:35.706780 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.706785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.706855 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.706966 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:35.708654 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.709718 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.711467 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.711900 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.711996 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.714972 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:35.714979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:35.714983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:35.715023 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:35.717977 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.718672 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.718676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.718678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.718786 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.719175 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.719180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.719184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.719420 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:35.719668 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.719948 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.720146 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":246,"threads_started":1}
I20260812 06:54:35.721355 11009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:42111
I20260812 06:54:35.723047 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:35.728796 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.723939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.17.69.126:40315)
0812 06:54:35.724080 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.724084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.724449 (+ 365us) server_negotiation.cc:407] Connection header received
0812 06:54:35.724604 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.724608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.724715 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.724818 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:35.725144 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.725756 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.727149 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.727544 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.727628 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.728008 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:35.728413 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:35.728514 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.728584 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.728638 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:54:35.729578 11009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:35.736615 11009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:35.759272 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.748306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34601 (local address 127.0.0.1:39594)
0812 06:54:35.749325 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.749342 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.749450 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.750908 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.750916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.750950 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:35.751378 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.751391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.753468 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.753474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.756028 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.756049 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.758383 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.758391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.758592 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.758642 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.758765 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.758940 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"thread_start_us":740,"threads_started":1}
I20260812 06:54:35.760280 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.748006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.17.69.66:34601)
0812 06:54:35.749963 (+ 1957us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.749973 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.750009 (+ 36us) server_negotiation.cc:407] Connection header received
0812 06:54:35.750108 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.750116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.750379 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.750567 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:35.751632 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.753264 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.756248 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.757857 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.757979 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.758213 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.759775 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.759947 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1782,"thread_start_us":1650,"threads_started":1}
I20260812 06:54:35.764511 11448 tablet_service.cc:1467] Processing CreateTablet for tablet db46198167c046738595809ee2910351 (DEFAULT_TABLE table=test-table [id=101c8c52c7eb4e81bb4578e4d832f7ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:35.766086 11448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46198167c046738595809ee2910351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:35.768412 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.749783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.17.69.65:33747)
0812 06:54:35.761727 (+ 11944us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.761736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.761765 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:54:35.761864 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.761872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.762086 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.762280 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:35.762634 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.764056 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.765908 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.767319 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.767523 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.767714 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.767935 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.768098 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11786,"thread_start_us":11672,"threads_started":1}
I20260812 06:54:35.768457 11834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.747353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37875 (local address 127.0.0.1:56040)
0812 06:54:35.748495 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.748545 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.748728 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.761444 (+ 12716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.761450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.761476 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:35.761797 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.761810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.763956 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.763961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.765931 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.765946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.767649 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.767660 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.767832 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.767863 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.767931 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.768059 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":1,"thread_start_us":713,"threads_started":1}
I20260812 06:54:35.768781 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.749406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54660)
0812 06:54:35.759504 (+ 10098us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.759521 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.759624 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.762290 (+ 2666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.762295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.762320 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:35.762489 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.762500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.764191 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.764195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.765766 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.765778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.767524 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.767530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.767734 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.767767 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.767878 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.768022 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9984,"thread_start_us":11772,"threads_started":1}
I20260812 06:54:35.769207 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.747633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.17.69.67:37875)
0812 06:54:35.760855 (+ 13222us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.760862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.760892 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:54:35.760983 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.760990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.761234 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.761975 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:35.762233 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.763808 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.766087 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.767400 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.768187 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.768422 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.768729 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.768887 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13059,"thread_start_us":13106,"threads_started":1}
I20260812 06:54:35.772392 11691 tablet_service.cc:1467] Processing CreateTablet for tablet db46198167c046738595809ee2910351 (DEFAULT_TABLE table=test-table [id=101c8c52c7eb4e81bb4578e4d832f7ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:35.772287 11216 tablet_service.cc:1467] Processing CreateTablet for tablet db46198167c046738595809ee2910351 (DEFAULT_TABLE table=test-table [id=101c8c52c7eb4e81bb4578e4d832f7ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:35.773604 11216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46198167c046738595809ee2910351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:35.773622 11691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db46198167c046738595809ee2910351. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:35.779006 11842 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Bootstrap starting.
I20260812 06:54:35.781759 11842 tablet_bootstrap.cc:654] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:35.782829 11842 log.cc:824] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:35.785261 11842 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: No bootstrap required, opened a new log
I20260812 06:54:35.785485 11842 ts_tablet_manager.cc:1397] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 06:54:35.788225 11845 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Bootstrap starting.
I20260812 06:54:35.791028 11846 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Bootstrap starting.
I20260812 06:54:35.791682 11845 tablet_bootstrap.cc:654] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:35.792868 11845 log.cc:824] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:35.793673 11846 tablet_bootstrap.cc:654] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:35.794775 11846 log.cc:824] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:35.794790 11842 raft_consensus.cc:348] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.795215 11842 raft_consensus.cc:374] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:35.795325 11842 raft_consensus.cc:729] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33533b18acf242b28778b277e14f1ebd, State: Initialized, Role: FOLLOWER
I20260812 06:54:35.795928 11845 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: No bootstrap required, opened a new log
I20260812 06:54:35.795953 11842 consensus_queue.cc:260] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.796198 11845 ts_tablet_manager.cc:1397] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 06:54:35.797155 11562 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:35.797794 11842 ts_tablet_manager.cc:1428] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260812 06:54:35.801537 11846 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: No bootstrap required, opened a new log
I20260812 06:54:35.801797 11846 ts_tablet_manager.cc:1397] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 06:54:35.808763 11852 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:35.809063 11852 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.810357 11846 raft_consensus.cc:348] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.810802 11846 raft_consensus.cc:374] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:35.810909 11846 raft_consensus.cc:729] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dc17dedc1c4aa2920090c0ad9f954f, State: Initialized, Role: FOLLOWER
I20260812 06:54:35.810657 11845 raft_consensus.cc:348] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.811105 11845 raft_consensus.cc:374] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:35.811201 11845 raft_consensus.cc:729] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dfbd42eb8b445d8f6d84d45c7ed420, State: Initialized, Role: FOLLOWER
I20260812 06:54:35.811868 11846 consensus_queue.cc:260] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.811924 11845 consensus_queue.cc:260] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.813565 11852 leader_election.cc:288] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747)
I20260812 06:54:35.814693 11818 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:35.818251 11337 heartbeater.cc:503] Master 127.17.69.126:40315 was elected leader, sending a full tablet report...
I20260812 06:54:35.818938 11846 ts_tablet_manager.cc:1428] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260812 06:54:35.819113 11845 ts_tablet_manager.cc:1428] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent starting tablet: real 0.023s user 0.008s sys 0.010s
I20260812 06:54:35.821967 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.811232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:53279 (local address 127.17.69.67:37875)
0812 06:54:35.811405 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.811410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.813570 (+ 2160us) server_negotiation.cc:407] Connection header received
0812 06:54:35.814556 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.814565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.814680 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.814738 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:35.815980 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.817081 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.819618 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.820804 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.820930 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.821000 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.821645 (+ 645us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.821741 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":16256}
I20260812 06:54:35.822681 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.811188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37875 (local address 127.17.69.66:53279)
0812 06:54:35.813466 (+ 2278us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.813488 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.814200 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.814943 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.814947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.814972 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:35.815315 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.815325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.817253 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.817259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.819417 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.819437 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.821267 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.821273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.821466 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.821499 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.822127 (+ 628us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.822275 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2147,"thread_start_us":1993,"threads_started":1}
I20260812 06:54:35.823577 11738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46198167c046738595809ee2910351" candidate_uuid: "33533b18acf242b28778b277e14f1ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" is_pre_election: true
I20260812 06:54:35.823977 11738 raft_consensus.cc:2454] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33533b18acf242b28778b277e14f1ebd in term 0.
I20260812 06:54:35.824618 11372 leader_election.cc:302] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f; no voters:
I20260812 06:54:35.824998 11852 raft_consensus.cc:2781] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:35.825150 11852 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:35.825266 11852 raft_consensus.cc:3037] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:35.825968 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.813938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.17.69.66:56577)
0812 06:54:35.819779 (+ 5841us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:35.819797 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:35.819968 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:35.820422 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:35.820426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:35.820451 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:35.820814 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.820825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.822172 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.822177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.823832 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:35.823846 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.825409 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.825414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:35.825548 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.825574 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:54:35.825642 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 06:54:35.825724 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5699,"thread_start_us":6045,"threads_started":1}
I20260812 06:54:35.826071 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.813999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:56577 (local address 127.17.69.65:33747)
0812 06:54:35.814326 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.814332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.819870 (+ 5538us) server_negotiation.cc:407] Connection header received
0812 06:54:35.820117 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.820122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.820188 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.820245 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:35.820981 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.821999 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.823978 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.825061 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.825203 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.825262 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.825710 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.825836 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":189}
I20260812 06:54:35.826874 11252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46198167c046738595809ee2910351" candidate_uuid: "33533b18acf242b28778b277e14f1ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" is_pre_election: true
I20260812 06:54:35.827343 11252 raft_consensus.cc:2454] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33533b18acf242b28778b277e14f1ebd in term 0.
I20260812 06:54:35.828207 11852 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.828732 11852 leader_election.cc:288] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [CANDIDATE]: Term 1 election: Requested vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747)
I20260812 06:54:35.829022 11738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46198167c046738595809ee2910351" candidate_uuid: "33533b18acf242b28778b277e14f1ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
I20260812 06:54:35.829121 11252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db46198167c046738595809ee2910351" candidate_uuid: "33533b18acf242b28778b277e14f1ebd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
I20260812 06:54:35.829181 11738 raft_consensus.cc:3037] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:35.829319 11252 raft_consensus.cc:3037] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:35.831326 11738 raft_consensus.cc:2454] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33533b18acf242b28778b277e14f1ebd in term 1.
I20260812 06:54:35.831420 11252 raft_consensus.cc:2454] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33533b18acf242b28778b277e14f1ebd in term 1.
I20260812 06:54:35.831686 11372 leader_election.cc:302] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f; no voters:
I20260812 06:54:35.831948 11852 raft_consensus.cc:2781] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:35.832353 11852 raft_consensus.cc:686] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 1 LEADER]: Becoming Leader. State: Replica: 33533b18acf242b28778b277e14f1ebd, State: Running, Role: LEADER
I20260812 06:54:35.832890 11852 consensus_queue.cc:237] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:35.836964 11008 catalog_manager.cc:5495] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd reported cstate change: term changed from 0 to 1, leader changed from <none> to 33533b18acf242b28778b277e14f1ebd (127.17.69.66). New cstate: current_term: 1 leader_uuid: "33533b18acf242b28778b277e14f1ebd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } health_report { overall_health: UNKNOWN } } }
I20260812 06:54:35.857802 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.852625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.17.69.66:34601)
0812 06:54:35.852806 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.852812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.852873 (+ 61us) server_negotiation.cc:407] Connection header received
0812 06:54:35.853096 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.853100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.853169 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.853285 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:35.853653 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.854334 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.855786 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.856194 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.856300 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.856761 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:35.857271 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:35.857396 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.857490 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.857566 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:54:35.865010 11738 raft_consensus.cc:1261] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Refusing update from remote peer 33533b18acf242b28778b277e14f1ebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:35.865015 11252 raft_consensus.cc:1261] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Refusing update from remote peer 33533b18acf242b28778b277e14f1ebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:35.865803 11864 consensus_queue.cc:1035] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:35.866148 11852 consensus_queue.cc:1035] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:35.875590 11868 mvcc.cc:205] Tried to move back new op lower bound from 7317576400334204928 to 7317576400215441408. Current Snapshot: MvccSnapshot[applied={T|T < 7317576400334204928}]
I20260812 06:54:35.876850 11869 mvcc.cc:205] Tried to move back new op lower bound from 7317576400334204928 to 7317576400215441408. Current Snapshot: MvccSnapshot[applied={T|T < 7317576400334204928}]
I20260812 06:54:35.888293 11008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:35.888561 11008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:35.891875 11008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Sending DeleteTablet for 3 replicas of tablet db46198167c046738595809ee2910351
I20260812 06:54:35.892818 11691 tablet_service.cc:1514] Processing DeleteTablet for tablet db46198167c046738595809ee2910351 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:35 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:56040
I20260812 06:54:35.893057 11216 tablet_service.cc:1514] Processing DeleteTablet for tablet db46198167c046738595809ee2910351 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:35 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:54660
I20260812 06:54:35.893136 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 11069
I20260812 06:54:35.893898 11448 tablet_service.cc:1514] Processing DeleteTablet for tablet db46198167c046738595809ee2910351 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:35 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:39594
I20260812 06:54:35.896536 11878 tablet_replica.cc:330] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: stopping tablet replica
I20260812 06:54:35.896863 11878 raft_consensus.cc:2229] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:35.897246 11878 raft_consensus.cc:2258] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:35.899196 11878 ts_tablet_manager.cc:1905] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:35.899748 11886 tablet_replica.cc:330] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: stopping tablet replica
I20260812 06:54:35.903278 11886 raft_consensus.cc:2229] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:35.903864 11886 raft_consensus.cc:2258] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:35.904830 11881 tablet_replica.cc:330] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: stopping tablet replica
I20260812 06:54:35.905187 11881 raft_consensus.cc:2229] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:35.905627 11881 raft_consensus.cc:2258] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:35.906291 11878 ts_tablet_manager.cc:1918] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:35.906489 11878 log.cc:1196] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/db46198167c046738595809ee2910351
I20260812 06:54:35.907027 11878 ts_tablet_manager.cc:1939] T db46198167c046738595809ee2910351 P 63dc17dedc1c4aa2920090c0ad9f954f: Deleting consensus metadata
I20260812 06:54:35.907634 11886 ts_tablet_manager.cc:1905] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:35.908728 10991 catalog_manager.cc:4841] TS 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): tablet db46198167c046738595809ee2910351 (table test-table [id=101c8c52c7eb4e81bb4578e4d832f7ed]) successfully deleted
I20260812 06:54:35.909919 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:35.894014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.17.69.65:33747)
0812 06:54:35.894188 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:35.894194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:35.894214 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:54:35.894378 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:35.894384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:35.894464 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:35.894573 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:35.896273 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.897470 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.899337 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:35.900021 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:35.900150 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:35.903330 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:35.903346 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:35.903362 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:35.903402 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:35.906778 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.907574 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.907582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.907585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.907716 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:35.908137 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:35.908142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:35.908146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:35.908635 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:35.908911 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:35.909299 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:54:35.909577 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:54:35.910967 11329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:35.919943 11886 ts_tablet_manager.cc:1918] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:35.920099 11886 log.cc:1196] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/db46198167c046738595809ee2910351
I20260812 06:54:35.920637 11886 ts_tablet_manager.cc:1939] T db46198167c046738595809ee2910351 P 33533b18acf242b28778b277e14f1ebd: Deleting consensus metadata
I20260812 06:54:35.922300 10989 catalog_manager.cc:4841] TS 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): tablet db46198167c046738595809ee2910351 (table test-table [id=101c8c52c7eb4e81bb4578e4d832f7ed]) successfully deleted
I20260812 06:54:35.998865 11881 ts_tablet_manager.cc:1905] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:35.999680 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 11069
I20260812 06:54:36.026942 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 11344
W20260812 06:54:36.027691 10989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.69.65:33747 (error 108)
I20260812 06:54:36.032624 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.032130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54676)
0812 06:54:36.032362 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.032472 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 06:54:36.032905 10989 catalog_manager.cc:4584] TS 92dfbd42eb8b445d8f6d84d45c7ed420 (127.17.69.65:33747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db46198167c046738595809ee2910351: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
I20260812 06:54:36.045122 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.028103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.17.69.66:34601)
0812 06:54:36.028291 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:36.028297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:36.028407 (+ 110us) server_negotiation.cc:407] Connection header received
0812 06:54:36.028628 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:36.028634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:36.028732 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:36.028842 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:36.030659 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:36.031898 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:36.033958 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:36.034694 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:36.034815 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:36.038170 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:36.038187 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:36.038206 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:36.038256 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:36.042374 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:36.043007 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:36.043017 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:36.043021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:36.043217 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:36.043562 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:36.043567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:36.043578 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:36.044042 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:36.044358 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:36.044508 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 06:54:36.044735 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:54:36.045775 11555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:36.086566 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.086068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54680)
0812 06:54:36.086277 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.086407 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:54:36.087579 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.087157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54690)
0812 06:54:36.087314 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.087398 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 06:54:36.147948 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.147398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54702)
0812 06:54:36.147647 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.147760 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 06:54:36.148998 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.148587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54712)
0812 06:54:36.148740 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.148826 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 06:54:36.153308 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 11344
I20260812 06:54:36.178540 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 11573
I20260812 06:54:36.191073 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.179302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.17.69.67:37875)
0812 06:54:36.179486 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:36.179491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:36.179509 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:36.179599 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:36.179603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:36.179668 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:36.179747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:36.180991 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:36.181741 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:36.183037 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:36.183434 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:36.183524 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:36.185998 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:36.186011 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:36.186026 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:36.186061 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:36.188586 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:36.189261 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:36.189267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:36.189270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:36.189404 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:36.189700 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:36.189702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:36.189704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:36.190033 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:36.190255 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:36.190614 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 06:54:36.190765 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:54:36.191736 11809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:36.229678 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.229266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54714)
0812 06:54:36.229438 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.229534 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 06:54:36.233290 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.230288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54716)
0812 06:54:36.233079 (+ 2791us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.233163 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2677}
I20260812 06:54:36.278241 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 11573
I20260812 06:54:36.298141 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 10961
I20260812 06:54:36.299165 11053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:36.407491 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:36.406871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33747 (local address 127.0.0.1:54720)
0812 06:54:36.407214 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:36.407331 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:33747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260812 06:54:36.407549 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 10961
I20260812 06:54:36.432320 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--webserver_interface=127.17.69.126
--webserver_port=39233
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:40315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:36.756515 11894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:36.756808 11894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:36.756924 11894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:36.764878 11894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:36.764976 11894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:36.765026 11894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:36.765081 11894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:36.765125 11894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:36.774384 11894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:40315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=39233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:36.775502 11894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:36.776625 11894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:36.787299 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:36.787351 11900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:36.787387 11901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:36.788324 11894 server_base.cc:1034] running on GCE node
I20260812 06:54:36.788996 11894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:36.790197 11894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:36.791427 11894 hybrid_clock.cc:648] HybridClock initialized: now 1786517676791371 us; error 39 us; skew 500 ppm
I20260812 06:54:36.791774 11894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:36 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517676, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:36.794677 11894 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:36.795960 11894 webserver.cc:460] Webserver started at http://127.17.69.126:39233/ using document root <none> and password file <none>
I20260812 06:54:36.796578 11894 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:36.796692 11894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:36.801656 11894 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:54:36.804317 11918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:36.805526 11894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:54:36.805678 11894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "c7d103b9f60f49b582378d50651040ce"
format_stamp: "Formatted at 2026-08-12 06:54:31 on dist-test-slave-6zbq"
server_key: "24e3ba6ad5fd44ed00dbff1473851c80"
server_key_iv: "102133609765db32ede8955102d0823c"
server_key_version: "encryptionkey@0"
I20260812 06:54:36.806624 11894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:36.827317 11894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:36.830802 11894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:36.831028 11894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:36.847818 11894 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:40315
I20260812 06:54:36.847852 12014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:40315 every 8 connection(s)
I20260812 06:54:36.849156 11894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:36.855468 12015 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:54:36.856397 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 11894
I20260812 06:54:36.857169 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:33747
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=33665
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:36.858572 12015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap starting.
I20260812 06:54:36.877537 12015 log.cc:824] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:36.893304 12015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:54:36.893765 12015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce: Bootstrap complete.
I20260812 06:54:36.902258 12015 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:36.903137 12015 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Initialized, Role: FOLLOWER
I20260812 06:54:36.903710 12015 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:36.903920 12015 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:36.904026 12015 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:36.904179 12015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:54:36.907797 12015 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:36.908242 12015 leader_election.cc:302] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d103b9f60f49b582378d50651040ce; no voters:
I20260812 06:54:36.908890 12015 leader_election.cc:288] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 06:54:36.908955 12020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:54:36.910612 12020 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [term 3 LEADER]: Becoming Leader. State: Replica: c7d103b9f60f49b582378d50651040ce, State: Running, Role: LEADER
I20260812 06:54:36.911149 12015 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:36.911227 12020 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } }
I20260812 06:54:36.913482 12021 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:36.913836 12021 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:36.914382 12022 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d103b9f60f49b582378d50651040ce. Latest consensus state: current_term: 3 leader_uuid: "c7d103b9f60f49b582378d50651040ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d103b9f60f49b582378d50651040ce" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 40315 } } }
I20260812 06:54:36.914657 12022 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d103b9f60f49b582378d50651040ce [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:36.917927 12026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:36.926450 12026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:36.927927 12026 catalog_manager.cc:1259] Loaded cluster ID: f7a68e44acbf416bbe26d8cb42514b2b
I20260812 06:54:36.928032 12026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 06:54:36.930156 12026 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd77df3f9ad google::LogMessage::Fail() at ??:0
@ 0x7fd77df4288a google::LogMessage::SendToLog() at ??:0
@ 0x7fd77df3f4de google::LogMessage::Flush() at ??:0
@ 0x7fd77df40a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd793972512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd79396f976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd7939f7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd7939f7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd780b8b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd780b8e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd780b8e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd780b69eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd783e436db start_thread at ??:0
@ 0x7fd77cdd961f clone at ??:0
W20260812 06:54:37.263360 12017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:37.263620 12017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:37.263736 12017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:37.271018 12017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:37.271129 12017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:37.271250 12017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:37.280606 12017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:33747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=33665
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:37.281731 12017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:37.282943 12017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:37.295073 12078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:37.295006 12082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:37.295109 12079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:37.295899 12017 server_base.cc:1034] running on GCE node
I20260812 06:54:37.296458 12017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:37.297272 12017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:37.298419 12017 hybrid_clock.cc:648] HybridClock initialized: now 1786517677298368 us; error 43 us; skew 500 ppm
I20260812 06:54:37.298694 12017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:37 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:37.301549 12017 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:37.302821 12017 webserver.cc:460] Webserver started at http://127.17.69.65:33665/ using document root <none> and password file <none>
I20260812 06:54:37.303406 12017 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:37.303521 12017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:37.308362 12017 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:54:37.311136 12093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:37.312315 12017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 06:54:37.312464 12017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "92dfbd42eb8b445d8f6d84d45c7ed420"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "7eb6acd20c51532782b5fcb3cb3ee9a7"
server_key_iv: "887a93e49a7b7a5740c42cfa6b8f4d05"
server_key_version: "encryptionkey@0"
I20260812 06:54:37.313370 12017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:37.335933 12017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:37.339663 12017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:37.339893 12017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:37.340767 12017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:37.342433 12106 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:54:37.347687 12017 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:54:37.347798 12017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 06:54:37.347900 12017 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 06:54:37.350852 12017 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 06:54:37.350931 12017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260812 06:54:37.351227 12106 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Bootstrap starting.
I20260812 06:54:37.358748 12106 log.cc:824] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:37.372293 12106 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:54:37.373090 12106 tablet_bootstrap.cc:492] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Bootstrap complete.
I20260812 06:54:37.373628 12106 ts_tablet_manager.cc:1397] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20260812 06:54:37.381184 12106 raft_consensus.cc:348] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:37.382251 12106 raft_consensus.cc:729] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dfbd42eb8b445d8f6d84d45c7ed420, State: Initialized, Role: FOLLOWER
I20260812 06:54:37.382926 12106 consensus_queue.cc:260] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:37.387671 12106 ts_tablet_manager.cc:1428] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
I20260812 06:54:37.390113 12017 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:33747
I20260812 06:54:37.390177 12295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:33747 every 8 connection(s)
I20260812 06:54:37.391207 12017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:37.395219 12300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.393526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:53297)
0812 06:54:37.394237 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.394483 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":285,"threads_started":1}
W20260812 06:54:37.395730 12297 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:40315 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:40315: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
I20260812 06:54:37.395978 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 12017
I20260812 06:54:37.396479 12300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.396210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:47997)
0812 06:54:37.396327 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.396388 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 06:54:37.396703 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:34601
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=41089
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:54:37.397269 12300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.396977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:53107)
0812 06:54:37.397087 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.397159 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 06:54:37.397468 12297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:54:37.709949 12301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:37.710193 12301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:37.710321 12301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:37.717147 12301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:37.717247 12301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:37.717381 12301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:37.726297 12301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:34601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=41089
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:37.727348 12301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:37.728510 12301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:37.740371 12312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:37.740419 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:37.740448 12317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:37.741122 12301 server_base.cc:1034] running on GCE node
I20260812 06:54:37.741627 12301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:37.742443 12301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:37.743634 12301 hybrid_clock.cc:648] HybridClock initialized: now 1786517677743589 us; error 42 us; skew 500 ppm
I20260812 06:54:37.743912 12301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:37 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:37.746696 12301 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:37.747936 12301 webserver.cc:460] Webserver started at http://127.17.69.66:41089/ using document root <none> and password file <none>
I20260812 06:54:37.748504 12301 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:37.748617 12301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:37.753405 12301 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:54:37.755889 12329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:37.757083 12301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:54:37.757229 12301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "33533b18acf242b28778b277e14f1ebd"
format_stamp: "Formatted at 2026-08-12 06:54:32 on dist-test-slave-6zbq"
server_key: "b6a331e30c66303297663d29307ec710"
server_key_iv: "7f1f845a5509726f682727f6a1a65677"
server_key_version: "encryptionkey@0"
I20260812 06:54:37.758106 12301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:37.779968 12301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:37.783638 12301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:37.783867 12301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:37.784792 12301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:37.786026 12301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:37.786137 12301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:37.786242 12301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:37.786311 12301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:37.816521 12301 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:34601
I20260812 06:54:37.816605 12537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:34601 every 8 connection(s)
I20260812 06:54:37.817903 12301 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:37.821923 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 12301
I20260812 06:54:37.822746 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.820637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:41471)
0812 06:54:37.821330 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.821593 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":264,"threads_started":1}
I20260812 06:54:37.822737 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:37875
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:40315
--webserver_port=34685
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:37.823388 12538 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:40315 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:40315: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
I20260812 06:54:37.824194 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.823881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:40951)
0812 06:54:37.824016 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.824082 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 06:54:37.824965 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:37.824679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:52311)
0812 06:54:37.824790 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:37.824862 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 06:54:37.825189 12538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:54:38.132576 12541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:38.132829 12541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:38.132929 12541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:38.139732 12541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:38.139820 12541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:38.139933 12541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:38.149092 12541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:37875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=34685
--tserver_master_addrs=127.17.69.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:38.150171 12541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:38.151353 12541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:38.163754 12554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:38.163705 12550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:38.163793 12548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:38.164453 12541 server_base.cc:1034] running on GCE node
I20260812 06:54:38.164954 12541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:38.165731 12541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:38.166911 12541 hybrid_clock.cc:648] HybridClock initialized: now 1786517678166872 us; error 30 us; skew 500 ppm
I20260812 06:54:38.167196 12541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:38 dist-test-slave-6zbq krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517678, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:38.169938 12541 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:38.171196 12541 webserver.cc:460] Webserver started at http://127.17.69.67:34685/ using document root <none> and password file <none>
I20260812 06:54:38.171762 12541 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:38.171869 12541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:38.176635 12541 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:54:38.179013 12567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:38.180137 12541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:38.180284 12541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "63dc17dedc1c4aa2920090c0ad9f954f"
format_stamp: "Formatted at 2026-08-12 06:54:33 on dist-test-slave-6zbq"
server_key: "91355df24a38d176b45ef0d089b58b72"
server_key_iv: "2a9aebedd236b93133905255e74b5365"
server_key_version: "encryptionkey@0"
I20260812 06:54:38.181174 12541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:38.214977 12541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:38.218520 12541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:38.218746 12541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:38.219602 12541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:38.220795 12541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:38.220888 12541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:38.220985 12541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:38.221053 12541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:38.252344 12541 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:37875
I20260812 06:54:38.252413 12756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:37875 every 8 connection(s)
I20260812 06:54:38.253705 12541 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:38.258107 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.256424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:51023)
0812 06:54:38.257119 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.257331 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":251,"threads_started":1}
W20260812 06:54:38.258816 12760 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:40315 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:40315: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
I20260812 06:54:38.259514 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 12541
I20260812 06:54:38.259567 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.259262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:45413)
0812 06:54:38.259403 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.259466 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 06:54:38.260398 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.260079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:53377)
0812 06:54:38.260215 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.260291 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 06:54:38.260713 12760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:54:38.398653 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.397897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:51079)
0812 06:54:38.398317 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.398457 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":180,"threads_started":1}
I20260812 06:54:38.826510 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.825672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:36207)
0812 06:54:38.826122 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.826226 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":216,"threads_started":1}
I20260812 06:54:38.920449 12768 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:38.920750 12768 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:38.922243 12768 leader_election.cc:288] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
Aug 12 06:54:38 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
Aug 12 06:54:38 dist-test-slave-6zbq krb5kdc[10035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:54:38.955243 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.922632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34601 (local address 127.17.69.65:59757)
0812 06:54:38.936137 (+ 13505us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.936151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:38.936505 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:38.939274 (+ 2769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:38.939278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:38.940004 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:38.940623 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:38.940645 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.942124 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.942129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:38.943477 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:38.943490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.944446 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.944455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:38.944689 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:38.945441 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:38.945457 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:38.947799 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:38.952348 (+ 4549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:38.952351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:38.952353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:38.952696 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:38.953173 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:38.953176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:38.953179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:38.953316 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:38.954097 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:38.954102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:38.954247 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 06:54:38.954380 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:54:38.954740 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13397,"mutex_wait_us":170,"spinlock_wait_cycles":30080,"thread_start_us":13056,"threads_started":1}
I20260812 06:54:38.955329 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.922757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:51565 (local address 127.17.69.67:37875)
0812 06:54:38.934959 (+ 12202us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:38.934980 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:38.936144 (+ 1164us) server_negotiation.cc:407] Connection header received
0812 06:54:38.938090 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:38.938099 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:38.938499 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:38.938773 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:38.940809 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.941922 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.943609 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.944207 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.944376 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:38.948350 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:38.948360 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:38.948375 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:38.948421 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:38.951713 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:38.952358 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:38.952364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:38.952369 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:38.952518 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:38.952985 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:38.952990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:38.952994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:38.953556 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:38.953747 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:38.954521 (+ 774us) server_negotiation.cc:299] Negotiation successful
0812 06:54:38.954918 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12001,"thread_start_us":12233,"threads_started":1}
I20260812 06:54:38.955329 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.922938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:59757 (local address 127.17.69.66:34601)
0812 06:54:38.935670 (+ 12732us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:38.935686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:38.936231 (+ 545us) server_negotiation.cc:407] Connection header received
0812 06:54:38.938716 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:38.938725 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:38.939010 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:38.939265 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:38.940801 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.941962 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.943610 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.944207 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.944369 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:38.948061 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:38.948068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:38.948080 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:38.948121 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:38.951715 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:38.952850 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:38.952858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:38.952865 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:38.953048 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:38.953423 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:38.953428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:38.953431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:38.953978 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:38.954127 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:38.954622 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 06:54:38.954945 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12550,"thread_start_us":12384,"threads_started":1}
I20260812 06:54:38.955264 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:38.922448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37875 (local address 127.17.69.65:51565)
0812 06:54:38.935968 (+ 13520us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:38.936001 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:38.936518 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:38.938914 (+ 2396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:38.938924 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:38.939833 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:38.940622 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:38.940645 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.942098 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.942102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:38.943476 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:38.943488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:38.944397 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:38.944403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:38.944694 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:38.945387 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:38.945409 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:38.948088 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:38.951835 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:38.951842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:38.951851 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:38.952249 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:38.952630 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:38.952639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:38.952642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:38.952879 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:38.953710 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:38.953715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:38.953963 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 06:54:38.954247 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 06:54:38.954865 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13376,"mutex_wait_us":295,"spinlock_wait_cycles":15744,"thread_start_us":13441,"threads_started":1}
W20260812 06:54:38.958241 12099 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): Not found: Tablet not found: db46198167c046738595809ee2910351
W20260812 06:54:38.958532 12098 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): Not found: Tablet not found: db46198167c046738595809ee2910351
I20260812 06:54:38.958673 12098 leader_election.cc:302] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92dfbd42eb8b445d8f6d84d45c7ed420; no voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f
I20260812 06:54:38.959064 12768 raft_consensus.cc:2726] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:54:39.262007 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:39.261197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:36959)
0812 06:54:39.261634 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:39.261758 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":197,"threads_started":1}
I20260812 06:54:39.399853 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:39.399444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:51961)
0812 06:54:39.399630 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:39.399715 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 06:54:39.827968 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:39.827313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:40355)
0812 06:54:39.827739 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:39.827820 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
I20260812 06:54:40.263397 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:40.262792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:49741)
0812 06:54:40.263177 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:40.263254 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":196,"threads_started":1}
I20260812 06:54:40.401391 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:40.400646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:51775)
0812 06:54:40.401126 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:40.401206 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":224,"threads_started":1}
I20260812 06:54:40.532493 12784 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:40.532694 12784 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:40.533337 12784 leader_election.cc:288] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
W20260812 06:54:40.534289 12099 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): Not found: Tablet not found: db46198167c046738595809ee2910351
W20260812 06:54:40.534502 12098 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): Not found: Tablet not found: db46198167c046738595809ee2910351
I20260812 06:54:40.534612 12098 leader_election.cc:302] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92dfbd42eb8b445d8f6d84d45c7ed420; no voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f
I20260812 06:54:40.534855 12784 raft_consensus.cc:2726] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:54:40.829345 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:40.828725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:36839)
0812 06:54:40.829110 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:40.829189 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":197,"threads_started":1}
I20260812 06:54:41.264739 12788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:41.264082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:55419)
0812 06:54:41.264506 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:41.264580 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":208,"threads_started":1}
I20260812 06:54:41.402832 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:41.402165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:45289)
0812 06:54:41.402601 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:41.402678 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":242,"threads_started":1}
I20260812 06:54:41.830848 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:41.830125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:59393)
0812 06:54:41.830588 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:41.830682 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":228,"threads_started":1}
I20260812 06:54:42.266294 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:42.265565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:53071)
0812 06:54:42.266010 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:42.266088 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":215,"threads_started":1}
I20260812 06:54:42.404414 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:42.403665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:53669)
0812 06:54:42.404144 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:42.404224 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":234,"threads_started":1}
I20260812 06:54:42.832350 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:42.831649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:60827)
0812 06:54:42.832089 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:42.832187 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260812 06:54:43.267838 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:43.267168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:56945)
0812 06:54:43.267610 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:43.267696 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260812 06:54:43.405824 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:43.405168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:50587)
0812 06:54:43.405585 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:43.405670 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260812 06:54:43.700508 12801 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:43.700692 12801 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:43.701368 12801 leader_election.cc:288] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
W20260812 06:54:43.702279 12098 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): Not found: Tablet not found: db46198167c046738595809ee2910351
W20260812 06:54:43.702481 12099 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): Not found: Tablet not found: db46198167c046738595809ee2910351
I20260812 06:54:43.702615 12099 leader_election.cc:302] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92dfbd42eb8b445d8f6d84d45c7ed420; no voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f
I20260812 06:54:43.702837 12801 raft_consensus.cc:2726] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:54:43.833809 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:43.833085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:54583)
0812 06:54:43.833540 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:43.833644 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260812 06:54:44.269358 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:44.268642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:57133)
0812 06:54:44.269119 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:44.269196 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260812 06:54:44.407274 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:44.406586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:58275)
0812 06:54:44.406989 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:44.407067 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":207,"threads_started":1}
I20260812 06:54:44.835376 12810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:44.834589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:49735)
0812 06:54:44.835104 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:44.835199 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":242,"threads_started":1}
I20260812 06:54:45.270967 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:45.270179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:35889)
0812 06:54:45.270698 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:45.270791 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":257,"threads_started":1}
I20260812 06:54:45.408675 12815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:45.408003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:49767)
0812 06:54:45.408407 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:45.408483 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":200,"threads_started":1}
I20260812 06:54:45.693070 12816 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:45.693261 12816 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:45.693923 12816 leader_election.cc:288] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
W20260812 06:54:45.694828 12099 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): Not found: Tablet not found: db46198167c046738595809ee2910351
W20260812 06:54:45.694989 12098 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): Not found: Tablet not found: db46198167c046738595809ee2910351
I20260812 06:54:45.695145 12098 leader_election.cc:302] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92dfbd42eb8b445d8f6d84d45c7ed420; no voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f
I20260812 06:54:45.695379 12816 raft_consensus.cc:2726] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:54:45.836772 12817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:45.836162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:38319)
0812 06:54:45.836525 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:45.836607 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":182,"threads_started":1}
I20260812 06:54:46.272373 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:46.271742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:54443)
0812 06:54:46.272128 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:46.272205 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":194,"threads_started":1}
I20260812 06:54:46.410041 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:46.409408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:60181)
0812 06:54:46.409810 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:46.409886 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":217,"threads_started":1}
I20260812 06:54:46.838296 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:46.837603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:37643)
0812 06:54:46.838058 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:46.838144 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":217,"threads_started":1}
I20260812 06:54:47.273847 12822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:47.273160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:54327)
0812 06:54:47.273612 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:47.273692 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":206,"threads_started":1}
I20260812 06:54:47.411507 12824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:47.410876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:36113)
0812 06:54:47.411278 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:47.411355 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":213,"threads_started":1}
I20260812 06:54:47.839777 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:47.839115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:33315)
0812 06:54:47.839531 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:47.839611 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260812 06:54:48.275442 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:48.274773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:47353)
0812 06:54:48.275175 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:48.275267 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":210,"threads_started":1}
I20260812 06:54:48.413026 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:48.412326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:32999)
0812 06:54:48.412769 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:48.412860 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":231,"threads_started":1}
I20260812 06:54:48.841274 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:48.840537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:48963)
0812 06:54:48.840997 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:48.841096 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":217,"threads_started":1}
I20260812 06:54:49.276917 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.276260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:55401)
0812 06:54:49.276678 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:49.276771 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":209,"threads_started":1}
I20260812 06:54:49.414542 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.413803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:42261)
0812 06:54:49.414264 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:49.414378 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":246,"threads_started":1}
I20260812 06:54:49.842795 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:49.842082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:46889)
0812 06:54:49.842541 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:49.842621 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":238,"threads_started":1}
I20260812 06:54:50.278432 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.277735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:55415)
0812 06:54:50.278139 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:50.278218 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":201,"threads_started":1}
I20260812 06:54:50.416031 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.415323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:34689)
0812 06:54:50.415774 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:50.415855 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":218,"threads_started":1}
I20260812 06:54:50.844244 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:50.843570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:57079)
0812 06:54:50.844012 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:50.844098 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":226,"threads_started":1}
I20260812 06:54:51.279932 12839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.279249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:43433)
0812 06:54:51.279692 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:51.279777 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":243,"threads_started":1}
I20260812 06:54:51.417527 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.416834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:58535)
0812 06:54:51.417271 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:51.417349 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":244,"threads_started":1}
I20260812 06:54:51.846760 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:51.845291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:44635)
0812 06:54:51.846220 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:51.846447 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":442,"threads_started":1}
I20260812 06:54:52.223412 12845 raft_consensus.cc:482] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:52.223590 12845 raft_consensus.cc:504] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dc17dedc1c4aa2920090c0ad9f954f" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37875 } } peers { permanent_uuid: "33533b18acf242b28778b277e14f1ebd" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34601 } } peers { permanent_uuid: "92dfbd42eb8b445d8f6d84d45c7ed420" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33747 } }
I20260812 06:54:52.224200 12845 leader_election.cc:288] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875), 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601)
W20260812 06:54:52.225081 12099 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63dc17dedc1c4aa2920090c0ad9f954f (127.17.69.67:37875): Not found: Tablet not found: db46198167c046738595809ee2910351
W20260812 06:54:52.225244 12098 leader_election.cc:341] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33533b18acf242b28778b277e14f1ebd (127.17.69.66:34601): Not found: Tablet not found: db46198167c046738595809ee2910351
I20260812 06:54:52.225356 12098 leader_election.cc:302] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92dfbd42eb8b445d8f6d84d45c7ed420; no voters: 33533b18acf242b28778b277e14f1ebd, 63dc17dedc1c4aa2920090c0ad9f954f
I20260812 06:54:52.225584 12845 raft_consensus.cc:2726] T db46198167c046738595809ee2910351 P 92dfbd42eb8b445d8f6d84d45c7ed420 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:54:52.281383 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.280665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:56309)
0812 06:54:52.281104 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.281197 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":223,"threads_started":1}
I20260812 06:54:52.419394 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.418606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.65:51769)
0812 06:54:52.419063 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.419188 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":202,"threads_started":1}
I20260812 06:54:52.848930 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:52.848182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.66:36151)
0812 06:54:52.848640 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:52.848755 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":216,"threads_started":1}
I20260812 06:54:53.264009 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 12017
I20260812 06:54:53.278923 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:53.265065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.17.69.65:33747)
0812 06:54:53.266191 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:53.266201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:53.266233 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:54:53.266295 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:53.266302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:53.266577 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:53.266727 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:53.268414 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.269412 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.270720 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.271274 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.271373 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:53.273785 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:53.273797 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:53.273810 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:53.273846 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:53.276493 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.277142 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.277149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.277151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.277288 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.277612 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.277617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.277619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.277987 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:53.278237 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:53.278458 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 06:54:53.278674 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":837,"threads_started":1}
I20260812 06:54:53.280076 12290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:53.284093 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:53.282383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:40315 (local address 127.17.69.67:50753)
0812 06:54:53.283796 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:53.283898 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:40315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1292,"thread_start_us":1159,"threads_started":1}
I20260812 06:54:53.366613 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 12017
I20260812 06:54:53.389863 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 12301
I20260812 06:54:53.402323 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:53.390525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.17.69.66:34601)
0812 06:54:53.390943 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:53.390948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:53.390969 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:54:53.391025 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:53.391028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:53.391088 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:53.391183 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:53.392416 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.393139 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.394482 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.394947 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.395061 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:53.397468 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:53.397473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:53.397477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:53.397509 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:53.399803 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.400464 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.400469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.400471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.400613 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.400960 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.400964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.400966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.401247 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:53.401486 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:53.401749 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 06:54:53.401958 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":211,"threads_started":1}
I20260812 06:54:53.402971 12525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:53.483675 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 12301
I20260812 06:54:53.509506 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 12541
I20260812 06:54:53.522449 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:53.510380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.17.69.67:37875)
0812 06:54:53.510937 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:53.510941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:53.510959 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:53.511014 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:53.511017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:53.511075 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:53.511193 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:53.512453 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.513261 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.514731 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:53.515137 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:53.515239 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:53.517734 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:53.517740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:53.517743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:53.517773 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:53.520159 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.520765 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.520768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.520771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.520906 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:53.521255 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:53.521258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:53.521260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:53.521496 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:53.521695 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:53.521889 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 06:54:53.522047 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":287,"threads_started":1}
I20260812 06:54:53.523138 12747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:53.604053 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 12541
2026-08-12T06:54:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23959 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:54:53 dist-test-slave-6zbq krb5kdc[12873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:54:53 dist-test-slave-6zbq krb5kdc[12873](info): set up 2 sockets
Aug 12 06:54:53 dist-test-slave-6zbq krb5kdc[12873](info): commencing operation
krb5kdc: starting...
W20260812 06:54:54.856660 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:54:55 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:54:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:54:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:54:55.099146 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:55.413278 12893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:55.413527 12893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:55.413614 12893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:55.420511 12893 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:54:55.420590 12893 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:54:55.420625 12893 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:54:55.420675 12893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:54:55.420725 12893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:55.420799 12893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:55.420841 12893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:54:55.420892 12893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:54:55.430070 12893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:55.431107 12893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:55.432253 12893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:55.443243 12903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:55.443272 12907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:55.443351 12904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:55.443831 12893 server_base.cc:1034] running on GCE node
I20260812 06:54:55.444579 12893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:55.445793 12893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:55.446995 12893 hybrid_clock.cc:648] HybridClock initialized: now 1786517695446932 us; error 37 us; skew 500 ppm
I20260812 06:54:55.447286 12893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:55 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:55.450145 12893 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:54:55.451524 12893 webserver.cc:460] Webserver started at http://127.17.69.126:37673/ using document root <none> and password file <none>
I20260812 06:54:55.452135 12893 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:55.452240 12893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:55.452517 12893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:55.455265 12893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "e88697aa44721c0164fa5d5c622d1eb1"
server_key_iv: "688d2fd70cd249fe5a817a218db3fe63"
server_key_version: "encryptionkey@0"
I20260812 06:54:55.455973 12893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "e88697aa44721c0164fa5d5c622d1eb1"
server_key_iv: "688d2fd70cd249fe5a817a218db3fe63"
server_key_version: "encryptionkey@0"
I20260812 06:54:55.460649 12893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:54:55.463838 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:55.465049 12893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:54:55.465211 12893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "e88697aa44721c0164fa5d5c622d1eb1"
server_key_iv: "688d2fd70cd249fe5a817a218db3fe63"
server_key_version: "encryptionkey@0"
I20260812 06:54:55.465415 12893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:55.482553 12893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:55.486362 12893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:55.486572 12893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:55.503911 12893 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:41415
I20260812 06:54:55.503935 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:41415 every 8 connection(s)
I20260812 06:54:55.505518 12893 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:54:55.508670 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 12893
I20260812 06:54:55.508888 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:54:55.509613 13012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:55.509723 17685 external_mini_cluster.cc:1428] Setting key c2acbd806e58362b4ed077764807349b
I20260812 06:54:55.523281 13012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap starting.
Aug 12 06:54:55 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:55.526628 13012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:55.527901 13012 log.cc:824] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:55.530483 13012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: No bootstrap required, opened a new log
I20260812 06:54:55.532207 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:55.511563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.17.69.126:41415)
0812 06:54:55.512301 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:55.512315 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:55.512363 (+ 48us) server_negotiation.cc:407] Connection header received
0812 06:54:55.514509 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:55.514537 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:55.514996 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:55.515406 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:55.516632 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.518567 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.519859 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:55.520906 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:55.521107 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:55.524820 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:55.524833 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:55.524850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:55.524890 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:55.528533 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.529162 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.529170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.529177 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.529341 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:55.529700 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:55.529705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:55.529709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:55.530177 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:55.530455 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:55.530915 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 06:54:55.531305 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":294,"threads_started":1}
I20260812 06:54:55.538576 13012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:54:55.538898 13012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:55.538983 13012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Initialized, Role: FOLLOWER
I20260812 06:54:55.539462 13012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:54:55.539669 13012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:54:55.539790 13012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:54:55.539932 13012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:55.542208 13012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:54:55.542656 13012 leader_election.cc:302] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 269b6ea81caf4a63a1fdc34e0c7bad6a; no voters:
I20260812 06:54:55.543157 13012 leader_election.cc:288] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:54:55.543272 13021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:55.543857 13021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 LEADER]: Becoming Leader. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Running, Role: LEADER
I20260812 06:54:55.544381 13021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:54:55.544886 13012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:54:55.546934 13023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 269b6ea81caf4a63a1fdc34e0c7bad6a. Latest consensus state: current_term: 1 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:54:55.548004 13023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:55.547705 13022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:54:55.548406 13022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:54:55.550561 13034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:54:55.554399 13034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:54:55.561573 13034 catalog_manager.cc:1347] Generated new cluster ID: 782cfe5124c042eeac6c502838979b2d
I20260812 06:54:55.561666 13034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:54:55.573267 13034 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:54:55.574476 13034 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:54:55.580245 13034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Generated new TSK 0
I20260812 06:54:55.580984 13034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:54:55.640913 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:55.959273 13052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:55.959509 13052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:55.959633 13052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:55.966629 13052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:55.966728 13052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:55.966871 13052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:54:55.976157 13052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:55.977370 13052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:55.978683 13052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:55.990394 13064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:55.990530 13059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:55.990541 13058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:55.991403 13052 server_base.cc:1034] running on GCE node
I20260812 06:54:55.991963 13052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:55.992825 13052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:55.994014 13052 hybrid_clock.cc:648] HybridClock initialized: now 1786517695993969 us; error 39 us; skew 500 ppm
I20260812 06:54:55.994289 13052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:55 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:55.997211 13052 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:54:55.998564 13052 webserver.cc:460] Webserver started at http://127.17.69.65:34147/ using document root <none> and password file <none>
I20260812 06:54:55.999150 13052 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:55.999272 13052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:55.999542 13052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:56.002254 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "785ee1401b1a4c98a0b29461e53f8f51"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "096157b4405743e1142e71da1223a439"
server_key_iv: "55dab5423c7582ec40a5eb1d3dec4ccf"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.002993 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "785ee1401b1a4c98a0b29461e53f8f51"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "096157b4405743e1142e71da1223a439"
server_key_iv: "55dab5423c7582ec40a5eb1d3dec4ccf"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.007627 13052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 06:54:56.010727 13072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.011847 13052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260812 06:54:56.011993 13052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "785ee1401b1a4c98a0b29461e53f8f51"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "096157b4405743e1142e71da1223a439"
server_key_iv: "55dab5423c7582ec40a5eb1d3dec4ccf"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.012173 13052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:56.030084 13052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:56.033696 13052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:56.033922 13052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:56.034808 13052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:56.036033 13052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:56.036104 13052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.036193 13052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:56.036242 13052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.065280 13052 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:46519
I20260812 06:54:56.065337 13278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:46519 every 8 connection(s)
I20260812 06:54:56.066462 13052 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:54:56.073534 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 13052
I20260812 06:54:56.073776 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:54:56.074596 17685 external_mini_cluster.cc:1428] Setting key 234b7d9e6a7d69cb3e045bf038098e13
Aug 12 06:54:56 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:56.085431 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:56.068687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:49405 (local address 127.17.69.126:41415)
0812 06:54:56.069130 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:56.069136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:56.070082 (+ 946us) server_negotiation.cc:407] Connection header received
0812 06:54:56.071871 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:56.071875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:56.071940 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:56.072036 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:56.073378 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.074488 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.076083 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.076597 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.076721 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:56.079844 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:56.079851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:56.079855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:56.079895 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:56.082700 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:56.083336 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:56.083342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:56.083344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:56.083461 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:56.083860 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:56.083865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:56.083868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:56.084122 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:56.084253 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:56.084853 (+ 600us) server_negotiation.cc:299] Negotiation successful
0812 06:54:56.085074 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260812 06:54:56.086248 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:56.069201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:49405)
0812 06:54:56.069942 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:56.069985 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:56.071654 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:56.072220 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:56.072227 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:56.072684 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:56.073218 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:56.073237 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.074632 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.074636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:56.075943 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:56.075955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.076816 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.076824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:56.077037 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:56.077609 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:56.077635 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:56.079628 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:56.082822 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:56.082830 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:56.082844 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:56.083224 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:56.083567 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:56.083572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:56.083575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:56.083754 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:56.084248 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:56.084256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:56.084533 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 06:54:56.084761 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 06:54:56.085221 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":298,"threads_started":1}
I20260812 06:54:56.089198 13279 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:54:56.089466 13279 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:56.090087 13279 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:54:56.092144 12951 ts_manager.cc:194] Registered new tserver with Master: 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519)
I20260812 06:54:56.094007 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:49405
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:54:56.133267 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:56.444446 13287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:56.444690 13287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:56.444814 13287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:56.451704 13287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:56.451803 13287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:56.451944 13287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:54:56.460867 13287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:56.461931 13287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:56.463114 13287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:56.474937 13298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:56.475006 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:56.475028 13301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:56.475677 13287 server_base.cc:1034] running on GCE node
I20260812 06:54:56.476168 13287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:56.476929 13287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:56.478137 13287 hybrid_clock.cc:648] HybridClock initialized: now 1786517696478087 us; error 41 us; skew 500 ppm
I20260812 06:54:56.478451 13287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:56 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517696, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:56.481330 13287 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:54:56.482635 13287 webserver.cc:460] Webserver started at http://127.17.69.66:34023/ using document root <none> and password file <none>
I20260812 06:54:56.483222 13287 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:56.483338 13287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:56.483608 13287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:56.486303 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "06d3b36ded474cecbf2031df1fc15cef"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "eaf70fbb22adcfd8cfa70e1a83cf4b76"
server_key_iv: "ba7f299c6881036c606096fae4f5e89e"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.487028 13287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "06d3b36ded474cecbf2031df1fc15cef"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "eaf70fbb22adcfd8cfa70e1a83cf4b76"
server_key_iv: "ba7f299c6881036c606096fae4f5e89e"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.491837 13287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:54:56.494997 13312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.496136 13287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 06:54:56.496284 13287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "06d3b36ded474cecbf2031df1fc15cef"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "eaf70fbb22adcfd8cfa70e1a83cf4b76"
server_key_iv: "ba7f299c6881036c606096fae4f5e89e"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.496457 13287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:56.515391 13287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:56.519004 13287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:56.519271 13287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:56.520089 13287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:56.521417 13287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:56.521507 13287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.521628 13287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:56.521683 13287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.551506 13287 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:41097
I20260812 06:54:56.551558 13518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:41097 every 8 connection(s)
I20260812 06:54:56.552644 13287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:54:56.554687 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 13287
I20260812 06:54:56.554867 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:54:56.555317 17685 external_mini_cluster.cc:1428] Setting key c0dd25910887e5f2e58d2430a9e5615c
Aug 12 06:54:56 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517696, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:56.572985 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:56.554905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:58799 (local address 127.17.69.126:41415)
0812 06:54:56.555112 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:56.555118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:56.556231 (+ 1113us) server_negotiation.cc:407] Connection header received
0812 06:54:56.558385 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:56.558391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:56.558466 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:56.558546 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:56.560354 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.561062 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.563009 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.563377 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.563467 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:56.567106 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:56.567112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:56.567117 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:56.567161 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:56.570110 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:56.570924 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:56.570928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:56.570930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:56.571058 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:56.571507 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:56.571510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:56.571512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:56.571749 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:56.571865 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:56.572577 (+ 712us) server_negotiation.cc:299] Negotiation successful
0812 06:54:56.572752 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:54:56.574149 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:56.555320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:58799)
0812 06:54:56.556059 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:56.556108 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:56.558106 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:56.558810 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:56.558821 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:56.559480 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:56.560197 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:56.560219 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.561222 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.561228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:56.562889 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:56.562903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:56.563595 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:56.563602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:56.563859 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:56.564526 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:56.564562 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:56.566897 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:56.570231 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:56.570240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:56.570255 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:56.570820 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:56.571173 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:56.571177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:56.571181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:56.571407 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:56.571867 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:56.571874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:56.572227 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 06:54:56.572486 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 06:54:56.572942 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":291,"threads_started":1}
I20260812 06:54:56.575418 13519 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:54:56.575688 13519 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:56.576319 13519 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:54:56.577644 12951 ts_manager.cc:194] Registered new tserver with Master: 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097)
I20260812 06:54:56.578429 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:58799
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:54:56.614970 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:54:56.927093 13527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:54:56.927332 13527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:54:56.927424 13527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:54:56.934110 13527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:54:56.934196 13527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:54:56.934310 13527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:54:56.943210 13527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:54:56.944193 13527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:54:56.945353 13527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:54:56.957021 13533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:56.956986 13534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:54:56.956941 13536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:54:56.957912 13527 server_base.cc:1034] running on GCE node
I20260812 06:54:56.958490 13527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:54:56.959331 13527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:54:56.960526 13527 hybrid_clock.cc:648] HybridClock initialized: now 1786517696960479 us; error 40 us; skew 500 ppm
I20260812 06:54:56.960804 13527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:54:56 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517696, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:54:56.963749 13527 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:54:56.964998 13527 webserver.cc:460] Webserver started at http://127.17.69.67:43209/ using document root <none> and password file <none>
I20260812 06:54:56.965654 13527 fs_manager.cc:359] Metadata directory not provided
I20260812 06:54:56.965773 13527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:54:56.966049 13527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:54:56.968894 13527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "dfe01958b7794b648c597e735ee24067"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "b986ec1038132b9399daad30108b6e7a"
server_key_iv: "71f00c3a5f11976a529edc36a7cffd89"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.969650 13527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "dfe01958b7794b648c597e735ee24067"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "b986ec1038132b9399daad30108b6e7a"
server_key_iv: "71f00c3a5f11976a529edc36a7cffd89"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.974283 13527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:54:56.977448 13551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:56.978636 13527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:54:56.978785 13527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "dfe01958b7794b648c597e735ee24067"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "b986ec1038132b9399daad30108b6e7a"
server_key_iv: "71f00c3a5f11976a529edc36a7cffd89"
server_key_version: "encryptionkey@0"
I20260812 06:54:56.978961 13527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:54:57.000537 13527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:54:57.004145 13527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:54:57.004385 13527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:54:57.005204 13527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:54:57.006494 13527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:54:57.006574 13527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.006690 13527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:54:57.006742 13527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:54:57.035976 13527 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:35821
I20260812 06:54:57.036028 13758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:35821 every 8 connection(s)
I20260812 06:54:57.037156 13527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:54:57.046540 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 13527
I20260812 06:54:57.046710 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:54:57.047300 17685 external_mini_cluster.cc:1428] Setting key 93acc63a123901b9b3f0871a3aa14450
Aug 12 06:54:57 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517696, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:54:57.055540 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.039465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:42845 (local address 127.17.69.126:41415)
0812 06:54:57.039691 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.039697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.040669 (+ 972us) server_negotiation.cc:407] Connection header received
0812 06:54:57.042419 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.042423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.042495 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.042587 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.043897 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.044630 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.046090 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.046627 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.046752 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.050509 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.050514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.050517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.050550 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.053087 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.053702 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.053705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.053708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.053827 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.054207 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.054209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.054211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.054491 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.054611 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.055145 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.055302 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:54:57.056356 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.039867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:42845)
0812 06:54:57.040531 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.040568 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.042157 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.042766 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.042773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.043230 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:54:57.043733 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.043755 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.044753 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.044756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.045948 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.045960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.046855 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.046863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.047148 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.047838 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:54:57.047861 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:54:57.050292 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:54:57.053189 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.053195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.053205 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.053620 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.053901 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:54:57.053904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:54:57.053906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:54:57.054124 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:54:57.054583 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:54:57.054588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:54:57.054861 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.055069 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.055413 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":260,"threads_started":1}
I20260812 06:54:57.057559 13759 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:54:57.057801 13759 heartbeater.cc:464] Registering TS with master...
I20260812 06:54:57.058470 13759 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:54:57.059765 12951 ts_manager.cc:194] Registered new tserver with Master: dfe01958b7794b648c597e735ee24067 (127.17.69.67:35821)
I20260812 06:54:57.060493 12951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:42845
I20260812 06:54:57.062484 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:54:57.078977 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.067099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.17.69.126:41415)
0812 06:54:57.067290 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.067296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.067758 (+ 462us) server_negotiation.cc:407] Connection header received
0812 06:54:57.067926 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.067929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.067983 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.068054 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.069863 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.070647 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.071965 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.072351 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.072438 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.074539 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.074543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.074546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.074575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.076854 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.077471 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.077474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.077476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.077586 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.077849 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.077852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.077854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.078110 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.078285 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.078563 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.078710 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:54:57.082566 12951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:54:57.089102 12951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:54:57.096668 13279 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:54:57.109355 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.100561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.17.69.66:41097)
0812 06:54:57.101096 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.101105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.101342 (+ 237us) server_negotiation.cc:407] Connection header received
0812 06:54:57.101744 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.101752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.101989 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.102172 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:57.103150 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.104479 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.106877 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.108240 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.108380 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.108565 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.108906 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.109072 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":257,"threads_started":1}
I20260812 06:54:57.109473 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.100229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41097 (local address 127.0.0.1:47932)
0812 06:54:57.101179 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.101216 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.101488 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.102462 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.102473 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.102512 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:57.102971 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.102984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.104607 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.104612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.106709 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.106732 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.108466 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.108473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.108680 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.108735 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.108843 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.109011 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":26,"spinlock_wait_cycles":1280,"thread_start_us":1524,"threads_started":1}
I20260812 06:54:57.110157 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.100570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:36998)
0812 06:54:57.101471 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.101489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.101623 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.102462 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.102473 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.102519 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:57.103583 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.103598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.105400 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.105405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.107277 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.107290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.109321 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.109332 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.109503 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.109539 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.109626 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.109744 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":550,"spinlock_wait_cycles":3328,"thread_start_us":648,"threads_started":1}
I20260812 06:54:57.110867 13780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.101011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.17.69.65:46519)
0812 06:54:57.101559 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.101567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.101592 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:54:57.101804 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.101810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.101987 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.102179 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:57.103744 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.105261 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.107428 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.108800 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.108942 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.109164 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.110396 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.110560 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":1704,"threads_started":1}
I20260812 06:54:57.111984 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.102040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:35821 (local address 127.0.0.1:48882)
0812 06:54:57.102633 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.102651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.102772 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.104929 (+ 2157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.104934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.104958 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:57.105380 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.105393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.107182 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.107187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.108606 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.108620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.111338 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.111342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.111452 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.111480 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.111555 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.111680 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":751,"threads_started":1}
I20260812 06:54:57.112087 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.102421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.17.69.67:35821)
0812 06:54:57.104159 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.104167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.104198 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:54:57.104283 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.104290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.104487 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.104671 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:57.105583 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.107028 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.108763 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.110953 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.111074 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.111232 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.111620 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.111776 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1572,"thread_start_us":1411,"threads_started":1}
I20260812 06:54:57.113472 13399 tablet_service.cc:1467] Processing CreateTablet for tablet ec48367948ae4d6caedc805702180aa4 (DEFAULT_TABLE table=test-table [id=c4120ec059ab41448d27c2d2b4697841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:57.114214 13160 tablet_service.cc:1467] Processing CreateTablet for tablet ec48367948ae4d6caedc805702180aa4 (DEFAULT_TABLE table=test-table [id=c4120ec059ab41448d27c2d2b4697841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:57.114979 13399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48367948ae4d6caedc805702180aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:57.115573 13160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48367948ae4d6caedc805702180aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:57.116992 13634 tablet_service.cc:1467] Processing CreateTablet for tablet ec48367948ae4d6caedc805702180aa4 (DEFAULT_TABLE table=test-table [id=c4120ec059ab41448d27c2d2b4697841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:54:57.118058 13634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48367948ae4d6caedc805702180aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:54:57.128648 13785 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Bootstrap starting.
I20260812 06:54:57.131003 13786 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Bootstrap starting.
I20260812 06:54:57.133227 13785 tablet_bootstrap.cc:654] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:57.134024 13789 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Bootstrap starting.
I20260812 06:54:57.134379 13786 tablet_bootstrap.cc:654] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:57.134748 13785 log.cc:824] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:57.135588 13786 log.cc:824] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:57.136835 13789 tablet_bootstrap.cc:654] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Neither blocks nor log segments found. Creating new log.
I20260812 06:54:57.137712 13789 log.cc:824] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Log is configured to *not* fsync() on all Append() calls
I20260812 06:54:57.137866 13785 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: No bootstrap required, opened a new log
I20260812 06:54:57.138142 13785 ts_tablet_manager.cc:1397] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 06:54:57.140532 13789 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: No bootstrap required, opened a new log
I20260812 06:54:57.140554 13786 tablet_bootstrap.cc:492] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: No bootstrap required, opened a new log
I20260812 06:54:57.140769 13786 ts_tablet_manager.cc:1397] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 06:54:57.140781 13789 ts_tablet_manager.cc:1397] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:54:57.147459 13786 raft_consensus.cc:348] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.147785 13786 raft_consensus.cc:374] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:57.147878 13786 raft_consensus.cc:729] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785ee1401b1a4c98a0b29461e53f8f51, State: Initialized, Role: FOLLOWER
I20260812 06:54:57.148017 13785 raft_consensus.cc:348] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.148412 13785 raft_consensus.cc:374] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:57.148502 13785 raft_consensus.cc:729] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe01958b7794b648c597e735ee24067, State: Initialized, Role: FOLLOWER
I20260812 06:54:57.148437 13786 consensus_queue.cc:260] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.149153 13785 consensus_queue.cc:260] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.150048 13786 ts_tablet_manager.cc:1428] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260812 06:54:57.150760 13789 raft_consensus.cc:348] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.151185 13789 raft_consensus.cc:374] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:54:57.151211 13759 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:54:57.151283 13789 raft_consensus.cc:729] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3b36ded474cecbf2031df1fc15cef, State: Initialized, Role: FOLLOWER
I20260812 06:54:57.151806 13789 consensus_queue.cc:260] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.154100 13785 ts_tablet_manager.cc:1428] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260812 06:54:57.155447 13519 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:54:57.156047 13789 ts_tablet_manager.cc:1428] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260812 06:54:57.288700 13760 tablet.cc:2369] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:57.304438 13521 tablet.cc:2369] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:54:57.318620 13280 tablet.cc:2369] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:54:57.407032 13797 raft_consensus.cc:482] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:54:57.407343 13797 raft_consensus.cc:504] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.408978 13797 leader_election.cc:288] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097), 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519)
I20260812 06:54:57.414516 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.409259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.17.69.67:37565)
0812 06:54:57.409541 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.409560 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.409689 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.410101 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.410106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.410197 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:57.410434 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.410444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.411421 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.411427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.412696 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.412709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.413799 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.413805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.413975 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.414000 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.414081 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.414204 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"spinlock_wait_cycles":21760}
I20260812 06:54:57.414511 13780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.409570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:37565 (local address 127.17.69.65:46519)
0812 06:54:57.409778 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.409784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.409806 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:54:57.409886 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.409891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.409968 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.410075 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:57.410573 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.411287 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.412856 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.413646 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.413792 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.413864 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.414120 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.414250 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:54:57.415033 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.409296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41097 (local address 127.17.69.67:41849)
0812 06:54:57.410005 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 06:54:57.410022 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:54:57.410139 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:54:57.410545 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:54:57.410549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:54:57.410573 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:54:57.410870 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.410879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.411871 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.411874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.413188 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:54:57.413201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.414319 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.414325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:54:57.414484 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.414509 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:54:57.414587 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:54:57.414712 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":799,"threads_started":1}
I20260812 06:54:57.414995 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.409458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:41849 (local address 127.17.69.66:41097)
0812 06:54:57.409807 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.409813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.410095 (+ 282us) server_negotiation.cc:407] Connection header received
0812 06:54:57.410359 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.410365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.410434 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.410512 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:54:57.411022 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.411708 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.413348 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.414116 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.414265 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.414335 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.414629 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.414754 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260812 06:54:57.415686 13196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48367948ae4d6caedc805702180aa4" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785ee1401b1a4c98a0b29461e53f8f51" is_pre_election: true
I20260812 06:54:57.416060 13430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48367948ae4d6caedc805702180aa4" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3b36ded474cecbf2031df1fc15cef" is_pre_election: true
I20260812 06:54:57.416296 13196 raft_consensus.cc:2454] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 0.
I20260812 06:54:57.416612 13430 raft_consensus.cc:2454] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 0.
I20260812 06:54:57.417007 13552 leader_election.cc:302] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 785ee1401b1a4c98a0b29461e53f8f51, dfe01958b7794b648c597e735ee24067; no voters:
I20260812 06:54:57.417397 13797 raft_consensus.cc:2781] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:54:57.417544 13797 raft_consensus.cc:482] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:54:57.417639 13797 raft_consensus.cc:3037] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:57.419816 13797 raft_consensus.cc:504] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.420409 13797 leader_election.cc:288] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 election: Requested vote from peers 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097), 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519)
I20260812 06:54:57.420743 13430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48367948ae4d6caedc805702180aa4" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3b36ded474cecbf2031df1fc15cef"
I20260812 06:54:57.420917 13430 raft_consensus.cc:3037] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:57.420884 13196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48367948ae4d6caedc805702180aa4" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785ee1401b1a4c98a0b29461e53f8f51"
I20260812 06:54:57.421065 13196 raft_consensus.cc:3037] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:54:57.423115 13430 raft_consensus.cc:2454] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 1.
I20260812 06:54:57.423225 13196 raft_consensus.cc:2454] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 1.
I20260812 06:54:57.423480 13554 leader_election.cc:302] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d3b36ded474cecbf2031df1fc15cef, dfe01958b7794b648c597e735ee24067; no voters:
I20260812 06:54:57.423738 13797 raft_consensus.cc:2781] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:54:57.424111 13797 raft_consensus.cc:686] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 1 LEADER]: Becoming Leader. State: Replica: dfe01958b7794b648c597e735ee24067, State: Running, Role: LEADER
I20260812 06:54:57.424535 13797 consensus_queue.cc:237] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } }
I20260812 06:54:57.428742 12946 catalog_manager.cc:5495] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfe01958b7794b648c597e735ee24067 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "dfe01958b7794b648c597e735ee24067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } health_report { overall_health: HEALTHY } } }
I20260812 06:54:57.477174 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.471878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.17.69.67:35821)
0812 06:54:57.472068 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.472074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.472093 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:54:57.472336 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.472340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.472427 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.472549 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:54:57.472908 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.473626 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.475116 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.475502 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.475617 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.476118 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:54:57.476603 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:54:57.476738 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.476842 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.476935 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:54:57.484828 13430 raft_consensus.cc:1261] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Refusing update from remote peer dfe01958b7794b648c597e735ee24067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:57.484828 13196 raft_consensus.cc:1261] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Refusing update from remote peer dfe01958b7794b648c597e735ee24067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:54:57.485828 13802 consensus_queue.cc:1035] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:57.486166 13797 consensus_queue.cc:1035] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:54:57.509912 12951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:57.510174 12951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:54:57.513603 12951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Sending DeleteTablet for 3 replicas of tablet ec48367948ae4d6caedc805702180aa4
I20260812 06:54:57.514619 13160 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48367948ae4d6caedc805702180aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:57 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:36998
I20260812 06:54:57.514962 13399 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48367948ae4d6caedc805702180aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:57 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:47932
I20260812 06:54:57.516302 13634 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48367948ae4d6caedc805702180aa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:54:57 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:48882
I20260812 06:54:57.518602 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 13052
I20260812 06:54:57.526803 13830 tablet_replica.cc:330] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: stopping tablet replica
I20260812 06:54:57.527223 13830 raft_consensus.cc:2229] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:57.527719 13830 raft_consensus.cc:2258] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:57.529886 13830 ts_tablet_manager.cc:1905] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:57.536742 13830 ts_tablet_manager.cc:1918] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:57.536901 13830 log.cc:1196] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/ec48367948ae4d6caedc805702180aa4
I20260812 06:54:57.537408 13830 ts_tablet_manager.cc:1939] T ec48367948ae4d6caedc805702180aa4 P 06d3b36ded474cecbf2031df1fc15cef: Deleting consensus metadata
I20260812 06:54:57.538022 13834 tablet_replica.cc:330] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: stopping tablet replica
I20260812 06:54:57.538226 13828 tablet_replica.cc:330] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: stopping tablet replica
I20260812 06:54:57.538654 13828 raft_consensus.cc:2229] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:54:57.539166 13828 raft_consensus.cc:2258] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:57.540293 12921 catalog_manager.cc:4841] TS 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097): tablet ec48367948ae4d6caedc805702180aa4 (table test-table [id=c4120ec059ab41448d27c2d2b4697841]) successfully deleted
I20260812 06:54:57.540802 13834 raft_consensus.cc:2229] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:54:57.541242 13828 ts_tablet_manager.cc:1905] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:57.541465 13834 raft_consensus.cc:2258] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:54:57.543962 13834 ts_tablet_manager.cc:1905] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:54:57.548318 13828 ts_tablet_manager.cc:1918] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:57.548470 13828 log.cc:1196] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/ec48367948ae4d6caedc805702180aa4
I20260812 06:54:57.548980 13828 ts_tablet_manager.cc:1939] T ec48367948ae4d6caedc805702180aa4 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting consensus metadata
I20260812 06:54:57.550724 12919 catalog_manager.cc:4841] TS 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519): tablet ec48367948ae4d6caedc805702180aa4 (table test-table [id=c4120ec059ab41448d27c2d2b4697841]) successfully deleted
Aug 12 06:54:57 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:54:57.551981 13834 ts_tablet_manager.cc:1918] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:54:57.552228 13834 log.cc:1196] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/ec48367948ae4d6caedc805702180aa4
I20260812 06:54:57.552845 13834 ts_tablet_manager.cc:1939] T ec48367948ae4d6caedc805702180aa4 P dfe01958b7794b648c597e735ee24067: Deleting consensus metadata
I20260812 06:54:57.554488 12921 catalog_manager.cc:4841] TS dfe01958b7794b648c597e735ee24067 (127.17.69.67:35821): tablet ec48367948ae4d6caedc805702180aa4 (table test-table [id=c4120ec059ab41448d27c2d2b4697841]) successfully deleted
I20260812 06:54:57.559243 13780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.519806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.17.69.65:46519)
0812 06:54:57.519978 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.519984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.537226 (+ 17242us) server_negotiation.cc:407] Connection header received
0812 06:54:57.538253 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.538260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.538389 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.538468 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.543096 (+ 4628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.544197 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.546152 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.546810 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.546910 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.552698 (+ 5788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.552712 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.552728 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.552768 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.556179 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.556885 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.556892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.556895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.557020 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.557395 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.557399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.557401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.557716 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.557997 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.558403 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.558711 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:54:57.560001 13274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:57.647867 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 13052
I20260812 06:54:57.683974 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 13287
Aug 12 06:54:57 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:54:57.701439 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.684826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.17.69.66:41097)
0812 06:54:57.684983 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.684987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.685005 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:54:57.685143 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.685147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.685211 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.685289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.687048 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.687757 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.689737 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.690143 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.690232 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.695817 (+ 5585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.695834 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.695851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.695891 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.698625 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.699373 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.699380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.699382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.699529 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.699972 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.699977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.699980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.700375 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.700620 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.700976 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.701153 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:54:57.702044 13511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:57.789578 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 13287
I20260812 06:54:57.812079 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 13527
Aug 12 06:54:57 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:54:57.830650 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:54:57.813010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.17.69.67:35821)
0812 06:54:57.813164 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:54:57.813169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:54:57.813186 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:54:57.813303 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:54:57.813307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:54:57.813374 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:54:57.813468 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:54:57.815531 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.816347 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.818190 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:54:57.818657 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:54:57.818747 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:54:57.824181 (+ 5434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:54:57.824201 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:54:57.824222 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:54:57.824275 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:54:57.827722 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.828524 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.828531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.828534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.828680 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:54:57.829121 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:54:57.829124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:54:57.829126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:54:57.829438 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:54:57.829683 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 06:54:57.830029 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 06:54:57.830192 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:54:57.831399 13752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:57.947645 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 13527
I20260812 06:54:57.973021 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 12893
I20260812 06:54:57.974045 13001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:54:58.078980 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 12893
I20260812 06:55:18.105784 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--webserver_interface=127.17.69.126
--webserver_port=37673
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:18.443670 13845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:18.443939 13845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:18.444073 13845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:18.451014 13845 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:55:18.451099 13845 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:55:18.451169 13845 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:55:18.451241 13845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:18.451311 13845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:18.451406 13845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:18.451463 13845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:18.451536 13845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:18.461018 13845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=37673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:18.462205 13845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:18.463515 13845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:18.487133 13854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:18.487042 13856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:18.487066 13859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:18.487527 13845 server_base.cc:1034] running on GCE node
I20260812 06:55:18.488441 13845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:18.489827 13845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:18.491046 13845 hybrid_clock.cc:648] HybridClock initialized: now 1786517718490973 us; error 47 us; skew 500 ppm
I20260812 06:55:18.491408 13845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:18 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:18.499085 13845 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:55:18.500658 13845 webserver.cc:460] Webserver started at http://127.17.69.126:37673/ using document root <none> and password file <none>
I20260812 06:55:18.501390 13845 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:18.501492 13845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:18.507113 13845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260812 06:55:18.509902 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:18.511415 13845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:18.511554 13845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "e88697aa44721c0164fa5d5c622d1eb1"
server_key_iv: "688d2fd70cd249fe5a817a218db3fe63"
server_key_version: "encryptionkey@0"
I20260812 06:55:18.512512 13845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:18.531612 13845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:18.535446 13845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:18.535692 13845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:18.553368 13845 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:41415
I20260812 06:55:18.553411 13953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:41415 every 8 connection(s)
I20260812 06:55:18.554774 13845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:18.557512 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 13845
I20260812 06:55:18.558396 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:46519
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=34147
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:18.561731 13954 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:55:18.565106 13954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap starting.
I20260812 06:55:18.589488 13954 log.cc:824] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:18.604990 13954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:55:18.605579 13954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap complete.
I20260812 06:55:18.618520 13954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:18.619633 13954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Initialized, Role: FOLLOWER
I20260812 06:55:18.620337 13954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:18.620617 13954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:18.620762 13954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:18.620942 13954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:55:18.626446 13954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:18.627060 13954 leader_election.cc:302] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 269b6ea81caf4a63a1fdc34e0c7bad6a; no voters:
I20260812 06:55:18.627660 13954 leader_election.cc:288] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 06:55:18.627781 13960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:55:18.628894 13960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 LEADER]: Becoming Leader. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Running, Role: LEADER
I20260812 06:55:18.629393 13960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:18.629765 13954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:18.631744 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 269b6ea81caf4a63a1fdc34e0c7bad6a. Latest consensus state: current_term: 2 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:55:18.632129 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:18.632614 13962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:55:18.632928 13962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:18.642483 13968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:18.649537 13968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:18.650905 13968 catalog_manager.cc:1259] Loaded cluster ID: 782cfe5124c042eeac6c502838979b2d
I20260812 06:55:18.650995 13968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:18.654817 13968 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:18.656263 13968 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Loaded TSK: 0
I20260812 06:55:18.700028 13968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Generated new TSK 1
I20260812 06:55:18.700459 13968 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 06:55:18.936478 13957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:18.936751 13957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:18.936888 13957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:18.943990 13957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:18.944101 13957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:18.944252 13957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:55:18.953598 13957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:46519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=34147
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:18.954880 13957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:18.956121 13957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:18.969997 14000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:18.970152 14003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:18.970661 14005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:18.971586 13957 server_base.cc:1034] running on GCE node
I20260812 06:55:18.972169 13957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:18.973060 13957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:18.974270 13957 hybrid_clock.cc:648] HybridClock initialized: now 1786517718974217 us; error 44 us; skew 500 ppm
I20260812 06:55:18.974594 13957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:18 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:18.977708 13957 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:55:18.979346 13957 webserver.cc:460] Webserver started at http://127.17.69.65:34147/ using document root <none> and password file <none>
I20260812 06:55:18.979995 13957 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:18.980129 13957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:18.985360 13957 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:55:18.988256 14020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:18.989591 13957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:55:18.989749 13957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "785ee1401b1a4c98a0b29461e53f8f51"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "096157b4405743e1142e71da1223a439"
server_key_iv: "55dab5423c7582ec40a5eb1d3dec4ccf"
server_key_version: "encryptionkey@0"
I20260812 06:55:18.990772 13957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:19.005034 13957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:19.008961 13957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:19.009224 13957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:19.010113 13957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:19.011372 13957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:19.011458 13957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.011565 13957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:19.011642 13957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.043607 13957 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:46519
I20260812 06:55:19.043680 14224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:46519 every 8 connection(s)
I20260812 06:55:19.044968 13957 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:19.053781 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 13957
I20260812 06:55:19.054406 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:41097
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=34023
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:55:19 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:55:19.066460 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.047795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:48045)
0812 06:55:19.048532 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:19.048568 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:19.050163 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:19.051353 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:19.051361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:19.051822 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:19.052334 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.052352 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.053877 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.053881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.055192 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.055206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.056220 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.056228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.056467 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.057082 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:19.057109 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:19.059301 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:19.062617 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.062625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.062638 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.063017 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.063438 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.063443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.063447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.063621 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.064472 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:19.064479 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:19.064793 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 06:55:19.065018 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 06:55:19.065436 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":304,"threads_started":1}
I20260812 06:55:19.067256 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.047604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:48045 (local address 127.17.69.126:41415)
0812 06:55:19.048191 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.048209 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.048709 (+ 500us) server_negotiation.cc:407] Connection header received
0812 06:55:19.050638 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.050651 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.050948 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.051217 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:19.052501 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.053700 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.055338 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.055969 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.056132 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.059554 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:19.059561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:19.059574 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:19.059609 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:19.062478 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.063151 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.063160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.063165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.063321 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.063736 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.063740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.063743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.064251 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:19.064407 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:19.065793 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0812 06:55:19.066197 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260812 06:55:19.069690 14227 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:55:19.069963 14227 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:19.070675 14227 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:55:19.072657 13901 ts_manager.cc:194] Registered new tserver with Master: 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519)
I20260812 06:55:19.074098 13901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:48045
W20260812 06:55:19.391561 14231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:19.391853 14231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:19.392005 14231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:19.399101 14231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:19.399212 14231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:19.399370 14231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:55:19.408713 14231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=34023
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:19.409929 14231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:19.411315 14231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:19.423995 14243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:19.423921 14246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:19.424024 14244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:19.425005 14231 server_base.cc:1034] running on GCE node
I20260812 06:55:19.425531 14231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:19.426450 14231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:19.427647 14231 hybrid_clock.cc:648] HybridClock initialized: now 1786517719427592 us; error 52 us; skew 500 ppm
I20260812 06:55:19.427940 14231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:19 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:19.431039 14231 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:55:19.432417 14231 webserver.cc:460] Webserver started at http://127.17.69.66:34023/ using document root <none> and password file <none>
I20260812 06:55:19.433013 14231 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:19.433126 14231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:19.438176 14231 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:19.440729 14256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.441969 14231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 06:55:19.442095 14231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "06d3b36ded474cecbf2031df1fc15cef"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "eaf70fbb22adcfd8cfa70e1a83cf4b76"
server_key_iv: "ba7f299c6881036c606096fae4f5e89e"
server_key_version: "encryptionkey@0"
I20260812 06:55:19.443066 14231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:19.477499 14231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:19.481572 14231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:19.481799 14231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:19.482705 14231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:19.484057 14231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:19.484153 14231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.484238 14231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:19.484297 14231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.514428 14231 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:41097
I20260812 06:55:19.514492 14460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:41097 every 8 connection(s)
I20260812 06:55:19.515736 14231 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:19.523229 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 14231
I20260812 06:55:19.523949 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:35821
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=43209
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 06:55:19 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:55:19.535176 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.518161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:33779 (local address 127.17.69.126:41415)
0812 06:55:19.518361 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.518367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.519415 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 06:55:19.521152 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.521157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.521228 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.521314 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:19.522733 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.523753 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.525315 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.525942 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.526061 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.529478 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:19.529484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:19.529488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:19.529552 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:19.532422 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.533073 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.533078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.533081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.533222 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.533597 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.533601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.533603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.533926 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:19.534060 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:19.534692 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 06:55:19.534898 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:55:19.535871 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.518559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:33779)
0812 06:55:19.519246 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:19.519282 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:19.520879 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:19.521511 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:19.521519 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:19.521998 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:19.522581 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.522608 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.523907 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.523910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.525152 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.525165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.526155 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.526160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.526387 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.527045 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:19.527071 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:19.529249 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:19.532542 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.532549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.532558 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.532945 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.533315 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.533318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.533320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.533490 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.534024 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:19.534029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:19.534307 (+ 278us) client_negotiation.cc:769] Sending connection context
0812 06:55:19.534563 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 06:55:19.534894 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260812 06:55:19.537217 14462 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:55:19.537483 14462 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:19.538178 14462 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:55:19.539613 13901 ts_manager.cc:194] Registered new tserver with Master: 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097)
I20260812 06:55:19.540396 13901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:33779
W20260812 06:55:19.847545 14466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:19.847779 14466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:19.847905 14466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:19.854700 14466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:19.854791 14466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:19.854935 14466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:55:19.863859 14466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:35821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=43209
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:19.864959 14466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:19.866098 14466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:19.877982 14473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:19.877940 14477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:19.878013 14479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:19.878659 14466 server_base.cc:1034] running on GCE node
I20260812 06:55:19.879186 14466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:19.879999 14466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:19.881198 14466 hybrid_clock.cc:648] HybridClock initialized: now 1786517719881151 us; error 37 us; skew 500 ppm
I20260812 06:55:19.881469 14466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:19 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:19.884680 14466 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:55:19.885969 14466 webserver.cc:460] Webserver started at http://127.17.69.67:43209/ using document root <none> and password file <none>
I20260812 06:55:19.886608 14466 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:19.886718 14466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:19.892261 14466 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:19.894982 14491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.896167 14466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:19.896307 14466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "dfe01958b7794b648c597e735ee24067"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "b986ec1038132b9399daad30108b6e7a"
server_key_iv: "71f00c3a5f11976a529edc36a7cffd89"
server_key_version: "encryptionkey@0"
I20260812 06:55:19.897189 14466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:19.911463 14466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:19.914829 14466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:19.915074 14466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:19.915948 14466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:19.917183 14466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:19.917266 14466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.917369 14466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:19.917433 14466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:19.947165 14466 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:35821
I20260812 06:55:19.947252 14700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:35821 every 8 connection(s)
I20260812 06:55:19.948504 14466 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:19.952344 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 14466
Aug 12 06:55:19 dist-test-slave-6zbq krb5kdc[12873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:55:19.968119 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.953401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.17.69.126:41415)
0812 06:55:19.954476 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.954481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.954502 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:19.954565 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.954570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.954649 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.954747 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:19.956443 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.957257 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.958649 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.959125 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.959229 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.962272 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:19.962278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:19.962281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:19.962310 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:19.965465 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.966139 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.966145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.966148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.966284 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.966684 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.966689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.966692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.966940 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:19.967212 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:19.967507 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 06:55:19.967710 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"spinlock_wait_cycles":22912,"thread_start_us":825,"threads_started":1}
I20260812 06:55:19.968467 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.950837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:58767 (local address 127.17.69.126:41415)
0812 06:55:19.951066 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.951073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.952321 (+ 1248us) server_negotiation.cc:407] Connection header received
0812 06:55:19.954638 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.954643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.954708 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.954826 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:19.956276 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.957140 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.958747 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.959171 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.959297 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.962757 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:19.962762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:19.962765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:19.962793 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:19.965658 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.966335 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.966361 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.966364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.966489 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.966912 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.966916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.966918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.967158 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:19.967282 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:19.967936 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 06:55:19.968136 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:55:19.969385 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.951301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:58767)
0812 06:55:19.952107 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:19.952165 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:19.954319 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:19.955068 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:19.955076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:19.955546 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:55:19.956080 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.956096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.957324 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.957330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.958596 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:19.958611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.959410 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.959418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:19.959622 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.960271 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:55:19.960292 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:55:19.962557 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:55:19.965796 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.965805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.965818 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.966208 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.966598 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:55:19.966603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:55:19.966607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:55:19.966787 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:55:19.967291 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:55:19.967298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:55:19.967586 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 06:55:19.967807 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 06:55:19.968289 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":329,"threads_started":1}
I20260812 06:55:19.970789 14703 heartbeater.cc:347] Connected to a master server at 127.17.69.126:41415
I20260812 06:55:19.971060 14703 heartbeater.cc:464] Registering TS with master...
I20260812 06:55:19.971729 14703 heartbeater.cc:511] Master 127.17.69.126:41415 requested a full tablet report, sending...
I20260812 06:55:19.972966 13901 ts_manager.cc:194] Registered new tserver with Master: dfe01958b7794b648c597e735ee24067 (127.17.69.67:35821)
I20260812 06:55:19.973748 13901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:58767
I20260812 06:55:19.982592 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:55:19.988561 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.983553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.17.69.126:41415)
0812 06:55:19.983707 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.983711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.984052 (+ 341us) server_negotiation.cc:407] Connection header received
0812 06:55:19.984226 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.984230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.984341 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.984456 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:19.984798 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.985541 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.987045 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.987407 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.987492 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.987908 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:19.988150 (+ 242us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 06:55:19.988401 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41462: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 06:55:19.988659 14229 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41462: token expired
W20260812 06:55:19.989223 13762 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.69.126:41415)
I20260812 06:55:20.000911 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:19.989684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.17.69.126:41415)
0812 06:55:19.989820 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:19.989823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:19.989838 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:55:19.989934 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:19.989937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:19.989984 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:19.990053 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:19.991410 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.992073 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.993309 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:19.993669 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:19.993794 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:19.996351 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:19.996356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:19.996359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:19.996386 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:19.998770 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.999310 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.999313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.999315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.999419 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:19.999712 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:19.999715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:19.999717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:19.999983 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.000177 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.000496 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.000668 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:55:20.003783 13901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:55:20.010821 13901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:55:20.044073 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.021391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45928)
0812 06:55:20.032622 (+ 11231us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.032679 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.032866 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.035280 (+ 2414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.035287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.035317 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:20.035738 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.035752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.038139 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.038145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.040816 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.040831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.042968 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.042974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.043283 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.043337 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.043462 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.043649 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10973,"mutex_wait_us":26,"thread_start_us":17296,"threads_started":1}
I20260812 06:55:20.044974 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.021547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.17.69.65:46519)
0812 06:55:20.034794 (+ 13247us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.034802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.034826 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:55:20.034908 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.034913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.035073 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.035252 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:20.035979 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.037959 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.040998 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.042373 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.042620 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.042806 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.044469 (+ 1663us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.044644 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13108,"thread_start_us":13014,"threads_started":1}
I20260812 06:55:20.046486 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.021896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.17.69.67:35821)
0812 06:55:20.034252 (+ 12356us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.034261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.036832 (+ 2571us) server_negotiation.cc:407] Connection header received
0812 06:55:20.037264 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.037273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.037484 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.037645 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:20.039472 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.040787 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.042560 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.044935 (+ 2375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.045031 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.045236 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.045830 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.045991 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12165,"thread_start_us":12353,"threads_started":1}
I20260812 06:55:20.046938 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.021626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:35821 (local address 127.0.0.1:48634)
0812 06:55:20.036714 (+ 15088us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.036732 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.037059 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.037949 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.037953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.037977 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:20.039306 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.039317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.040923 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.040927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.042379 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.042393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.045436 (+ 3043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.045442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.045601 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.045633 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.046393 (+ 760us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.046537 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14961,"mutex_wait_us":1,"thread_start_us":10601,"threads_started":1}
I20260812 06:55:20.047749 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.033173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41097 (local address 127.0.0.1:40656)
0812 06:55:20.039002 (+ 5829us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.039022 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.039151 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.039756 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.039760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.039788 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:20.040256 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.040270 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.042067 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.042072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.044079 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.044094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.047025 (+ 2931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.047032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.047216 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.047249 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.047338 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.047459 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5704,"thread_start_us":3675,"threads_started":1}
I20260812 06:55:20.048592 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.033581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.17.69.66:41097)
0812 06:55:20.039094 (+ 5513us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.039101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.039134 (+ 33us) server_negotiation.cc:407] Connection header received
0812 06:55:20.039342 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.039350 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.039538 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.039738 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:20.040429 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.041916 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.044286 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.045712 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.045833 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.048080 (+ 2247us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.048155 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.048310 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5365,"thread_start_us":5275,"threads_started":1}
I20260812 06:55:20.051977 14341 tablet_service.cc:1467] Processing CreateTablet for tablet 24b7027f07d44092be4b63bfd8158900 (DEFAULT_TABLE table=test-table [id=88da162a83074bc4aa9bcb2013e33d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:20.052024 14597 tablet_service.cc:1467] Processing CreateTablet for tablet 24b7027f07d44092be4b63bfd8158900 (DEFAULT_TABLE table=test-table [id=88da162a83074bc4aa9bcb2013e33d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:20.049213 14121 tablet_service.cc:1467] Processing CreateTablet for tablet 24b7027f07d44092be4b63bfd8158900 (DEFAULT_TABLE table=test-table [id=88da162a83074bc4aa9bcb2013e33d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:55:20.053156 14597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b7027f07d44092be4b63bfd8158900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:20.053196 14341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b7027f07d44092be4b63bfd8158900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:20.053346 14121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b7027f07d44092be4b63bfd8158900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:20.064283 14742 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Bootstrap starting.
I20260812 06:55:20.067428 14742 tablet_bootstrap.cc:654] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:20.067781 14749 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Bootstrap starting.
I20260812 06:55:20.068818 14742 log.cc:824] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:20.069216 14744 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Bootstrap starting.
I20260812 06:55:20.071048 14749 tablet_bootstrap.cc:654] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:20.072192 14749 log.cc:824] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:20.072249 14744 tablet_bootstrap.cc:654] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:20.073153 14742 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: No bootstrap required, opened a new log
I20260812 06:55:20.073410 14742 ts_tablet_manager.cc:1397] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260812 06:55:20.073457 14744 log.cc:824] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:20.075429 14749 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: No bootstrap required, opened a new log
I20260812 06:55:20.075747 14749 ts_tablet_manager.cc:1397] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260812 06:55:20.075809 14744 tablet_bootstrap.cc:492] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: No bootstrap required, opened a new log
I20260812 06:55:20.076076 14744 ts_tablet_manager.cc:1397] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:55:20.076922 14227 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:55:20.083977 14744 raft_consensus.cc:348] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.084363 14744 raft_consensus.cc:374] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:20.084463 14744 raft_consensus.cc:729] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3b36ded474cecbf2031df1fc15cef, State: Initialized, Role: FOLLOWER
I20260812 06:55:20.085013 14744 consensus_queue.cc:260] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.086073 14462 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:55:20.085914 14742 raft_consensus.cc:348] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.085914 14749 raft_consensus.cc:348] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.086352 14749 raft_consensus.cc:374] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:20.086316 14742 raft_consensus.cc:374] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:20.086449 14742 raft_consensus.cc:729] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe01958b7794b648c597e735ee24067, State: Initialized, Role: FOLLOWER
I20260812 06:55:20.086493 14749 raft_consensus.cc:729] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785ee1401b1a4c98a0b29461e53f8f51, State: Initialized, Role: FOLLOWER
I20260812 06:55:20.087007 14742 consensus_queue.cc:260] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.087064 14749 consensus_queue.cc:260] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.087911 14744 ts_tablet_manager.cc:1428] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260812 06:55:20.089718 14742 ts_tablet_manager.cc:1428] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Time spent starting tablet: real 0.016s user 0.005s sys 0.008s
I20260812 06:55:20.090118 14703 heartbeater.cc:503] Master 127.17.69.126:41415 was elected leader, sending a full tablet report...
I20260812 06:55:20.090138 14749 ts_tablet_manager.cc:1428] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260812 06:55:20.200017 14704 tablet.cc:2369] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:20.217849 14759 raft_consensus.cc:482] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:55:20.218150 14759 raft_consensus.cc:504] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.219919 14759 leader_election.cc:288] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519), 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097)
I20260812 06:55:20.226060 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.220389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:39083 (local address 127.17.69.65:46519)
0812 06:55:20.220594 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.220600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.220617 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:55:20.220691 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.220696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.220772 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.220860 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:20.221333 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.222270 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.224072 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.225144 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.225277 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.225333 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.225715 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.225816 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:55:20.226073 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.220113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.17.69.67:39083)
0812 06:55:20.220348 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.220372 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.220496 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.220924 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.220930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.220970 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:20.221159 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.221171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.222411 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.222417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.223905 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.223917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.225337 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.225343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.225531 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.225566 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.225663 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.225795 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 06:55:20.226761 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.220508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:37167 (local address 127.17.69.66:41097)
0812 06:55:20.220706 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.220713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.221208 (+ 495us) server_negotiation.cc:407] Connection header received
0812 06:55:20.221315 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.221321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.221384 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.221451 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:55:20.222004 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.222897 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.224817 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.225774 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.225938 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.226005 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.226474 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.226572 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:55:20.226822 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.220262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41097 (local address 127.17.69.67:37167)
0812 06:55:20.221068 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.221109 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:55:20.221209 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:55:20.221495 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:55:20.221499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:55:20.221525 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:55:20.221865 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.221877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.223041 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.223044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.224631 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:55:20.224646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.226002 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.226008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:55:20.226212 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.226245 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:55:20.226420 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 06:55:20.226542 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":477,"threads_started":1}
I20260812 06:55:20.227257 14152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b7027f07d44092be4b63bfd8158900" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785ee1401b1a4c98a0b29461e53f8f51" is_pre_election: true
I20260812 06:55:20.227728 14152 raft_consensus.cc:2454] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 0.
I20260812 06:55:20.227772 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b7027f07d44092be4b63bfd8158900" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3b36ded474cecbf2031df1fc15cef" is_pre_election: true
I20260812 06:55:20.228251 14375 raft_consensus.cc:2454] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 0.
I20260812 06:55:20.228364 14492 leader_election.cc:302] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 785ee1401b1a4c98a0b29461e53f8f51, dfe01958b7794b648c597e735ee24067; no voters:
I20260812 06:55:20.228734 14759 raft_consensus.cc:2781] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:55:20.228865 14759 raft_consensus.cc:482] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:55:20.228965 14759 raft_consensus.cc:3037] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:20.231299 14759 raft_consensus.cc:504] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.231928 14759 leader_election.cc:288] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 election: Requested vote from peers 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519), 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097)
I20260812 06:55:20.232276 14152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b7027f07d44092be4b63bfd8158900" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785ee1401b1a4c98a0b29461e53f8f51"
I20260812 06:55:20.232384 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b7027f07d44092be4b63bfd8158900" candidate_uuid: "dfe01958b7794b648c597e735ee24067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3b36ded474cecbf2031df1fc15cef"
I20260812 06:55:20.232462 14152 raft_consensus.cc:3037] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:20.232577 14375 raft_consensus.cc:3037] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:20.234894 14375 raft_consensus.cc:2454] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 1.
I20260812 06:55:20.235260 14495 leader_election.cc:302] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d3b36ded474cecbf2031df1fc15cef, dfe01958b7794b648c597e735ee24067; no voters:
I20260812 06:55:20.235458 14152 raft_consensus.cc:2454] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe01958b7794b648c597e735ee24067 in term 1.
I20260812 06:55:20.235522 14759 raft_consensus.cc:2781] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:20.236073 14759 raft_consensus.cc:686] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 1 LEADER]: Becoming Leader. State: Replica: dfe01958b7794b648c597e735ee24067, State: Running, Role: LEADER
I20260812 06:55:20.236502 14759 consensus_queue.cc:237] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } }
I20260812 06:55:20.240859 13900 catalog_manager.cc:5495] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfe01958b7794b648c597e735ee24067 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "dfe01958b7794b648c597e735ee24067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfe01958b7794b648c597e735ee24067" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 } health_report { overall_health: UNKNOWN } } }
I20260812 06:55:20.264974 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.259792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48644 (local address 127.17.69.67:35821)
0812 06:55:20.259994 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.260000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.260102 (+ 102us) server_negotiation.cc:407] Connection header received
0812 06:55:20.260318 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.260322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.260394 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.260513 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:55:20.260894 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.261586 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.262977 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.263367 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.263473 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.263984 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:55:20.264429 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:55:20.264567 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.264645 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.264742 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 06:55:20.267647 14463 tablet.cc:2369] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:55:20.272590 14375 raft_consensus.cc:1261] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Refusing update from remote peer dfe01958b7794b648c597e735ee24067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:20.272590 14152 raft_consensus.cc:1261] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Refusing update from remote peer dfe01958b7794b648c597e735ee24067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:55:20.273576 14764 consensus_queue.cc:1035] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "785ee1401b1a4c98a0b29461e53f8f51" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:20.273953 14759 consensus_queue.cc:1035] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d3b36ded474cecbf2031df1fc15cef" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:55:20.282682 14767 mvcc.cc:205] Tried to move back new op lower bound from 7317576582226722816 to 7317576582092591104. Current Snapshot: MvccSnapshot[applied={T|T < 7317576582226722816}]
I20260812 06:55:20.296320 13900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:20.296604 13900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:55:20.299968 13900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Sending DeleteTablet for 3 replicas of tablet 24b7027f07d44092be4b63bfd8158900
I20260812 06:55:20.301013 14121 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b7027f07d44092be4b63bfd8158900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:20 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:45928
I20260812 06:55:20.301204 14341 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b7027f07d44092be4b63bfd8158900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:20 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:40656
I20260812 06:55:20.301193 14597 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b7027f07d44092be4b63bfd8158900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:55:20 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:48634
I20260812 06:55:20.302445 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 13957
I20260812 06:55:20.311481 14783 tablet_replica.cc:330] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: stopping tablet replica
I20260812 06:55:20.312057 14783 raft_consensus.cc:2229] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:55:20.312704 14782 tablet_replica.cc:330] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: stopping tablet replica
I20260812 06:55:20.312932 14781 tablet_replica.cc:330] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: stopping tablet replica
I20260812 06:55:20.313082 14782 raft_consensus.cc:2229] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:20.313287 14781 raft_consensus.cc:2229] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:55:20.313584 14782 raft_consensus.cc:2258] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:20.313750 14781 raft_consensus.cc:2258] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:20.314417 14783 raft_consensus.cc:2258] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:55:20.316385 14782 ts_tablet_manager.cc:1905] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:20.316393 14781 ts_tablet_manager.cc:1905] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:20.320739 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.303366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.17.69.65:46519)
0812 06:55:20.303538 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.303544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.303565 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:55:20.303664 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.303669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.303756 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.303862 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.305632 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.306928 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.308909 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.309555 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.309671 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.313291 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.313307 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.313325 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.313376 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.317595 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.318440 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.318448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.318451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.318606 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.319022 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.319027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.319030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.319514 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.319806 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.320129 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.320360 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:55:20.321707 14219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:20.323169 14782 ts_tablet_manager.cc:1918] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:20.323333 14782 log.cc:1196] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/24b7027f07d44092be4b63bfd8158900
I20260812 06:55:20.323776 14782 ts_tablet_manager.cc:1939] T 24b7027f07d44092be4b63bfd8158900 P 06d3b36ded474cecbf2031df1fc15cef: Deleting consensus metadata
I20260812 06:55:20.325170 13878 catalog_manager.cc:4841] TS 06d3b36ded474cecbf2031df1fc15cef (127.17.69.66:41097): tablet 24b7027f07d44092be4b63bfd8158900 (table test-table [id=88da162a83074bc4aa9bcb2013e33d16]) successfully deleted
I20260812 06:55:20.326926 14783 ts_tablet_manager.cc:1905] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:20.334141 14783 ts_tablet_manager.cc:1918] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:20.334326 14783 log.cc:1196] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/24b7027f07d44092be4b63bfd8158900
I20260812 06:55:20.334887 14783 ts_tablet_manager.cc:1939] T 24b7027f07d44092be4b63bfd8158900 P dfe01958b7794b648c597e735ee24067: Deleting consensus metadata
I20260812 06:55:20.336505 13878 catalog_manager.cc:4841] TS dfe01958b7794b648c597e735ee24067 (127.17.69.67:35821): tablet 24b7027f07d44092be4b63bfd8158900 (table test-table [id=88da162a83074bc4aa9bcb2013e33d16]) successfully deleted
I20260812 06:55:20.415817 14781 ts_tablet_manager.cc:1918] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:20.416069 14781 log.cc:1196] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/24b7027f07d44092be4b63bfd8158900
I20260812 06:55:20.416527 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 13957
I20260812 06:55:20.438912 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 14231
W20260812 06:55:20.438930 13874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.69.65:46519 (error 108)
I20260812 06:55:20.441470 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.440968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45942)
0812 06:55:20.441171 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.441298 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 06:55:20.441838 13874 catalog_manager.cc:4584] TS 785ee1401b1a4c98a0b29461e53f8f51 (127.17.69.65:46519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24b7027f07d44092be4b63bfd8158900: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
I20260812 06:55:20.454531 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.439758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.17.69.66:41097)
0812 06:55:20.439937 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.439943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.440186 (+ 243us) server_negotiation.cc:407] Connection header received
0812 06:55:20.440282 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.440287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.440372 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.440467 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.442512 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.443672 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.444883 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.445517 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.445647 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.448252 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.448268 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.448294 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.448347 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.451771 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.452398 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.452406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.452410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.452561 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.452875 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.452880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.452884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.453353 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.453660 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.453904 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.454138 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:55:20.455204 14454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:20.495368 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.494884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45958)
0812 06:55:20.495093 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.495210 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:55:20.496533 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.496137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45972)
0812 06:55:20.496293 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.496380 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 06:55:20.543061 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 14231
I20260812 06:55:20.557044 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.556618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45974)
0812 06:55:20.556788 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.556883 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 06:55:20.558532 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.557920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45988)
0812 06:55:20.558116 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.558246 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 06:55:20.576176 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 14466
I20260812 06:55:20.589975 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.576937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.17.69.67:35821)
0812 06:55:20.577139 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:20.577145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:20.577208 (+ 63us) server_negotiation.cc:407] Connection header received
0812 06:55:20.577441 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:20.577446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:20.577513 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:20.577600 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:20.579018 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.579830 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.581165 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:20.581614 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:20.581755 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:20.584216 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:20.584231 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:20.584246 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:20.584280 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:20.587250 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.587873 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.587881 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.587883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.588041 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:20.588434 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:20.588438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:20.588440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:20.588802 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:20.589100 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:20.589423 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 06:55:20.589626 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 06:55:20.590687 14695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:20.639737 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.639237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:45990)
0812 06:55:20.639456 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.639560 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 06:55:20.640756 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.640383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:46000)
0812 06:55:20.640573 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.640653 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 06:55:20.711781 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 14466
I20260812 06:55:20.738835 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 13845
I20260812 06:55:20.739956 13947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:20.849859 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:20.849154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:46519 (local address 127.0.0.1:46004)
0812 06:55:20.849480 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:20.849612 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.65:46519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260812 06:55:20.850088 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 13845
I20260812 06:55:40.876253 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--webserver_interface=127.17.69.126
--webserver_port=37673
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:41415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:41.202175 14794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:41.202469 14794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:41.202589 14794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:41.209503 14794 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 06:55:41.209583 14794 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 06:55:41.209622 14794 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 06:55:41.209690 14794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:41.209756 14794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:41.209842 14794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:41.209896 14794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:41.209967 14794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:41.219780 14794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:41415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:41415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=37673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:41.220994 14794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:41.222412 14794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:41.233359 14803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.233392 14804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.233574 14807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:41.234238 14794 server_base.cc:1034] running on GCE node
I20260812 06:55:41.234995 14794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:41.236231 14794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:41.237461 14794 hybrid_clock.cc:648] HybridClock initialized: now 1786517741237392 us; error 44 us; skew 500 ppm
I20260812 06:55:41.237747 14794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:41 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:41.244129 14794 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:55:41.245445 14794 webserver.cc:460] Webserver started at http://127.17.69.126:37673/ using document root <none> and password file <none>
I20260812 06:55:41.246078 14794 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:41.246179 14794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:41.251255 14794 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 06:55:41.253988 14818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.255211 14794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:41.255385 14794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "e88697aa44721c0164fa5d5c622d1eb1"
server_key_iv: "688d2fd70cd249fe5a817a218db3fe63"
server_key_version: "encryptionkey@0"
I20260812 06:55:41.256299 14794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:41.267805 14794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:41.271749 14794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:41.271999 14794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:41.288636 14794 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:41415
I20260812 06:55:41.288666 14909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:41415 every 8 connection(s)
I20260812 06:55:41.289928 14794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:41.293876 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 14794
I20260812 06:55:41.294770 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:46519
--local_ip_for_outbound_sockets=127.17.69.65
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=34147
--webserver_interface=127.17.69.65
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:41.296366 14911 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:55:41.299360 14911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap starting.
I20260812 06:55:41.318477 14911 log.cc:824] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:55:41.333941 14911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:55:41.334478 14911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Bootstrap complete.
I20260812 06:55:41.342971 14911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:41.343798 14911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Initialized, Role: FOLLOWER
I20260812 06:55:41.344378 14911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:41.344581 14911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:41.344696 14911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:41.344853 14911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:55:41.348412 14911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:41.348850 14911 leader_election.cc:302] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 269b6ea81caf4a63a1fdc34e0c7bad6a; no voters:
I20260812 06:55:41.349449 14911 leader_election.cc:288] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 06:55:41.349586 14922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 3 FOLLOWER]: Leader election won for term 3
I20260812 06:55:41.350735 14922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [term 3 LEADER]: Becoming Leader. State: Replica: 269b6ea81caf4a63a1fdc34e0c7bad6a, State: Running, Role: LEADER
I20260812 06:55:41.351284 14922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } }
I20260812 06:55:41.351580 14911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:41.353657 14924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 269b6ea81caf4a63a1fdc34e0c7bad6a. Latest consensus state: current_term: 3 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:55:41.354022 14924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:41.354550 14923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "269b6ea81caf4a63a1fdc34e0c7bad6a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 41415 } } }
I20260812 06:55:41.354841 14923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a [sys.catalog]: This master's current role is: LEADER
I20260812 06:55:41.367985 14931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:55:41.376322 14931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:55:41.377877 14931 catalog_manager.cc:1259] Loaded cluster ID: 782cfe5124c042eeac6c502838979b2d
I20260812 06:55:41.378033 14931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:55:41.382031 14931 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:55:41.383666 14931 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Loaded TSK: 0
I20260812 06:55:41.383884 14931 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 269b6ea81caf4a63a1fdc34e0c7bad6a: Loaded TSK: 1
F20260812 06:55:41.385109 14931 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fac5bbb99ad google::LogMessage::Fail() at ??:0
@ 0x7fac5bbbc88a google::LogMessage::SendToLog() at ??:0
@ 0x7fac5bbb94de google::LogMessage::Flush() at ??:0
@ 0x7fac5bbbaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fac5f91d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fac5f90c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fac715e81c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fac715e7783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fac716732a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fac716730c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fac716bd5e9 std::function<>::operator()() at ??:0
@ 0x7fac715ebcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fac715e9c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fac71671610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fac71671439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fac5e805345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fac5e80891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fac5e808799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fac5e7e3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fac61abd6db start_thread at ??:0
@ 0x7fac5aa5361f clone at ??:0
W20260812 06:55:41.676302 14917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:41.676542 14917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:41.676663 14917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:41.683600 14917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:41.683694 14917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:41.683849 14917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:55:41.692870 14917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:46519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=34147
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:41.693921 14917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:41.695147 14917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:41.707237 15002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.707208 15003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:41.707216 15005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:41.707988 14917 server_base.cc:1034] running on GCE node
I20260812 06:55:41.708506 14917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:41.709332 14917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:41.710521 14917 hybrid_clock.cc:648] HybridClock initialized: now 1786517741710479 us; error 33 us; skew 500 ppm
I20260812 06:55:41.710809 14917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:41 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517741, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:41.713689 14917 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:55:41.715080 14917 webserver.cc:460] Webserver started at http://127.17.69.65:34147/ using document root <none> and password file <none>
I20260812 06:55:41.715696 14917 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:41.715814 14917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:41.720734 14917 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 06:55:41.723295 15018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.724486 14917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:55:41.724638 14917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "785ee1401b1a4c98a0b29461e53f8f51"
format_stamp: "Formatted at 2026-08-12 06:54:55 on dist-test-slave-6zbq"
server_key: "096157b4405743e1142e71da1223a439"
server_key_iv: "55dab5423c7582ec40a5eb1d3dec4ccf"
server_key_version: "encryptionkey@0"
I20260812 06:55:41.725541 14917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:41.742846 14917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:41.746727 14917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:41.746951 14917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:41.747866 14917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:41.749545 15032 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 06:55:41.754590 15032 ts_tablet_manager.cc:1852] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 06:55:41.755538 15032 ts_tablet_manager.cc:1905] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:55:41.762380 15032 ts_tablet_manager.cc:1918] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:55:41.762646 15032 ts_tablet_manager.cc:1939] T 24b7027f07d44092be4b63bfd8158900 P 785ee1401b1a4c98a0b29461e53f8f51: Deleting consensus metadata
I20260812 06:55:41.763037 14917 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 06:55:41.763149 14917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260812 06:55:41.763258 14917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:41.763351 14917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:41.794011 14917 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:46519
I20260812 06:55:41.794078 15225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:46519 every 8 connection(s)
I20260812 06:55:41.795203 14917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:55:41.799652 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.797880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:35819)
0812 06:55:41.798653 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.798888 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":294,"threads_started":1}
W20260812 06:55:41.800209 15226 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:41415 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:41415: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
I20260812 06:55:41.800915 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.800630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:45241)
0812 06:55:41.800753 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.800815 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 06:55:41.801637 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:41.801357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:58137)
0812 06:55:41.801459 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:41.801527 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 06:55:41.801849 15226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:55:41.803568 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 14917
I20260812 06:55:41.804193 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:41097
--local_ip_for_outbound_sockets=127.17.69.66
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=34023
--webserver_interface=127.17.69.66
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:42.129231 15230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:42.129503 15230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:42.129634 15230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:42.136600 15230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:42.136693 15230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:42.136835 15230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:55:42.145898 15230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=34023
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:42.146998 15230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:42.148165 15230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:42.159938 15239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.159968 15244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.160012 15241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:42.160964 15230 server_base.cc:1034] running on GCE node
I20260812 06:55:42.161517 15230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:42.162325 15230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:42.163550 15230 hybrid_clock.cc:648] HybridClock initialized: now 1786517742163500 us; error 39 us; skew 500 ppm
I20260812 06:55:42.163825 15230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:42 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:42.167203 15230 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:55:42.168431 15230 webserver.cc:460] Webserver started at http://127.17.69.66:34023/ using document root <none> and password file <none>
I20260812 06:55:42.169000 15230 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:42.169097 15230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:42.173844 15230 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260812 06:55:42.176312 15257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.177464 15230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:55:42.177605 15230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "06d3b36ded474cecbf2031df1fc15cef"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "eaf70fbb22adcfd8cfa70e1a83cf4b76"
server_key_iv: "ba7f299c6881036c606096fae4f5e89e"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.178514 15230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:42.191087 15230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:42.194605 15230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:42.194819 15230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:42.195669 15230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:42.196842 15230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:42.196934 15230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.197029 15230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:42.197095 15230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.225910 15230 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:41097
I20260812 06:55:42.225976 15451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:41097 every 8 connection(s)
I20260812 06:55:42.227245 15230 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:55:42.230818 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 15230
I20260812 06:55:42.231462 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:35821
--local_ip_for_outbound_sockets=127.17.69.67
--tserver_master_addrs=127.17.69.126:41415
--webserver_port=43209
--webserver_interface=127.17.69.67
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:55:42.231714 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.229915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:46833)
0812 06:55:42.230629 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.230840 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":249,"threads_started":1}
W20260812 06:55:42.232327 15453 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:41415 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:41415: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
I20260812 06:55:42.233131 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.232809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:51305)
0812 06:55:42.232947 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.233010 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 06:55:42.233925 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.233621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:60995)
0812 06:55:42.233753 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.233821 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 06:55:42.234143 15453 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 06:55:42.557958 15458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:42.558229 15458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:42.558378 15458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:42.565730 15458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:42.565840 15458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:42.565989 15458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:55:42.575029 15458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:35821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=43209
--tserver_master_addrs=127.17.69.126:41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:42.576092 15458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:42.577373 15458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:42.589232 15467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.589392 15469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:42.589422 15472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:42.590222 15458 server_base.cc:1034] running on GCE node
I20260812 06:55:42.590781 15458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:42.591621 15458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:42.592808 15458 hybrid_clock.cc:648] HybridClock initialized: now 1786517742592768 us; error 31 us; skew 500 ppm
I20260812 06:55:42.593098 15458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:42 dist-test-slave-6zbq krb5kdc[12873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517742, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:42.596009 15458 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:55:42.597297 15458 webserver.cc:460] Webserver started at http://127.17.69.67:43209/ using document root <none> and password file <none>
I20260812 06:55:42.597882 15458 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:42.597995 15458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:42.602784 15458 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 06:55:42.606935 15487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.608525 15458 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:55:42.608678 15458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "dfe01958b7794b648c597e735ee24067"
format_stamp: "Formatted at 2026-08-12 06:54:56 on dist-test-slave-6zbq"
server_key: "b986ec1038132b9399daad30108b6e7a"
server_key_iv: "71f00c3a5f11976a529edc36a7cffd89"
server_key_version: "encryptionkey@0"
I20260812 06:55:42.609881 15458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:42.651412 15458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:42.654881 15458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:42.655103 15458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:42.655974 15458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:55:42.657181 15458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:55:42.657289 15458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.657387 15458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:55:42.657462 15458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:42.686769 15458 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:35821
I20260812 06:55:42.686856 15672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:35821 every 8 connection(s)
I20260812 06:55:42.688088 15458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:55:42.690092 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 15458
I20260812 06:55:42.692862 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.690936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:47951)
0812 06:55:42.691698 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.692053 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":342,"threads_started":1}
W20260812 06:55:42.693456 15673 heartbeater.cc:644] Failed to heartbeat to 127.17.69.126:41415 (0 consecutive failures): Network error: Failed to ping master at 127.17.69.126:41415: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
I20260812 06:55:42.694155 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.693868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:51883)
0812 06:55:42.693993 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.694056 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 06:55:42.694903 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.694630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:56005)
0812 06:55:42.694731 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.694802 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 06:55:42.695120 15673 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 06:55:42.803134 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:42.802403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:50213)
0812 06:55:42.802823 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:42.802903 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":192,"threads_started":1}
I20260812 06:55:43.235306 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.234582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:42781)
0812 06:55:43.234988 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.235083 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":191,"threads_started":1}
I20260812 06:55:43.696278 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.695547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:35241)
0812 06:55:43.695969 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.696069 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":202,"threads_started":1}
I20260812 06:55:43.804560 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:43.803925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:52725)
0812 06:55:43.804322 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:43.804400 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":203,"threads_started":1}
I20260812 06:55:44.236706 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.236043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:47533)
0812 06:55:44.236462 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.236538 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":209,"threads_started":1}
I20260812 06:55:44.697687 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.697041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:39081)
0812 06:55:44.697452 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.697528 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":205,"threads_started":1}
I20260812 06:55:44.806023 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:44.805334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:43299)
0812 06:55:44.805771 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:44.805859 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":207,"threads_started":1}
I20260812 06:55:45.238106 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.237457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:59051)
0812 06:55:45.237866 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.237943 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":208,"threads_started":1}
I20260812 06:55:45.699105 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.698470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:50643)
0812 06:55:45.698857 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.698933 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":205,"threads_started":1}
I20260812 06:55:45.807500 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:45.806796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:52997)
0812 06:55:45.807234 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:45.807311 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":200,"threads_started":1}
I20260812 06:55:46.239530 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.238890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:53769)
0812 06:55:46.239274 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.239349 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":185,"threads_started":1}
I20260812 06:55:46.700529 15694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.699860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:48097)
0812 06:55:46.700293 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.700372 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":208,"threads_started":1}
I20260812 06:55:46.808967 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:46.808267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:40947)
0812 06:55:46.808730 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:46.808811 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":216,"threads_started":1}
I20260812 06:55:47.240962 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.240312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:43815)
0812 06:55:47.240728 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.240805 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":200,"threads_started":1}
I20260812 06:55:47.702051 15699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.701366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:48993)
0812 06:55:47.701810 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.701889 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":217,"threads_started":1}
I20260812 06:55:47.810539 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:47.809800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:37367)
0812 06:55:47.810253 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:47.810335 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
I20260812 06:55:48.242444 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.241721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:53285)
0812 06:55:48.242150 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.242251 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":208,"threads_started":1}
I20260812 06:55:48.703470 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.702778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:53379)
0812 06:55:48.703232 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.703320 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":223,"threads_started":1}
I20260812 06:55:48.812057 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:48.811296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:48031)
0812 06:55:48.811780 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:48.811861 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":217,"threads_started":1}
I20260812 06:55:49.243991 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.243206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:55419)
0812 06:55:49.243699 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.243786 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":203,"threads_started":1}
I20260812 06:55:49.705010 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.704252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:55529)
0812 06:55:49.704737 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.704815 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":207,"threads_started":1}
I20260812 06:55:49.813587 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:49.812866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:39641)
0812 06:55:49.813302 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:49.813411 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260812 06:55:50.245445 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.244772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:35161)
0812 06:55:50.245189 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.245286 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":211,"threads_started":1}
I20260812 06:55:50.706583 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.705861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:48667)
0812 06:55:50.706301 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.706416 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":247,"threads_started":1}
I20260812 06:55:50.815071 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:50.814383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:50771)
0812 06:55:50.814835 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:50.814922 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":215,"threads_started":1}
I20260812 06:55:51.246917 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.246208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:33957)
0812 06:55:51.246660 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.246756 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260812 06:55:51.707985 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.707324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:36849)
0812 06:55:51.707742 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.707826 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":203,"threads_started":1}
I20260812 06:55:51.816493 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:51.815839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:57401)
0812 06:55:51.816242 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:51.816321 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":206,"threads_started":1}
I20260812 06:55:52.248394 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.247705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:46315)
0812 06:55:52.248146 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.248228 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":233,"threads_started":1}
I20260812 06:55:52.709479 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.708825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:57557)
0812 06:55:52.709245 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.709325 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":231,"threads_started":1}
I20260812 06:55:52.818248 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:52.817375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:43475)
0812 06:55:52.817908 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:52.818017 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":267,"threads_started":1}
I20260812 06:55:53.249823 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.249188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:38059)
0812 06:55:53.249573 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.249656 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":190,"threads_started":1}
I20260812 06:55:53.710944 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.710237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:36063)
0812 06:55:53.710687 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.710774 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":240,"threads_started":1}
I20260812 06:55:53.819823 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:53.819161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:54371)
0812 06:55:53.819574 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:53.819657 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260812 06:55:54.251216 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.250599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:46795)
0812 06:55:54.250998 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:54.251073 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":202,"threads_started":1}
I20260812 06:55:54.712337 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.711720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:53653)
0812 06:55:54.712109 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:54.712190 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":193,"threads_started":1}
I20260812 06:55:54.821362 15746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:54.820654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:56871)
0812 06:55:54.821084 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:54.821188 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":205,"threads_started":1}
I20260812 06:55:55.252677 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.251974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:35477)
0812 06:55:55.252410 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:55.252500 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":211,"threads_started":1}
I20260812 06:55:55.713981 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.713124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:51775)
0812 06:55:55.713642 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:55.713764 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260812 06:55:55.822834 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:55.822102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:42009)
0812 06:55:55.822543 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:55.822642 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":195,"threads_started":1}
I20260812 06:55:56.254135 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.253438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:50791)
0812 06:55:56.253875 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.253955 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260812 06:55:56.715564 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.714811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:50331)
0812 06:55:56.715315 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.715395 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":267,"threads_started":1}
I20260812 06:55:56.824497 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:56.823794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.65:36271)
0812 06:55:56.824244 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:56.824332 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260812 06:55:57.256191 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.255252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.66:45559)
0812 06:55:57.255827 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:57.255945 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":281,"threads_started":1}
I20260812 06:55:57.699728 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 14917
I20260812 06:55:57.717362 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.716632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:41415 (local address 127.17.69.67:45085)
0812 06:55:57.717085 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 06:55:57.717174 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.126:41415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":242,"threads_started":1}
I20260812 06:55:57.718663 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.701624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.17.69.65:46519)
0812 06:55:57.702324 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.702368 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.702414 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:55:57.704068 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.704080 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.704488 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.704813 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.706303 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.707599 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.708922 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.709592 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.709765 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.712433 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.712441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.712455 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.712493 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.715380 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.716015 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.716024 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.716032 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.716261 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.716707 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.716710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.716712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.717248 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.717480 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.717940 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.718322 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":405,"threads_started":1}
I20260812 06:55:57.719954 15216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:57.803416 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 14917
I20260812 06:55:57.824827 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 15230
I20260812 06:55:57.845664 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.825658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45148 (local address 127.17.69.66:41097)
0812 06:55:57.826173 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.826190 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.826228 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:55:57.827735 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.827744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.828157 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.828470 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.829902 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.831295 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.832527 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.833190 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.833360 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.840026 (+ 6666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.840034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.840047 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.840090 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.843006 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.843602 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.843607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.843613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.843770 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.844058 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.844061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.844063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.844524 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.844740 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.845084 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.845391 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260812 06:55:57.846954 15446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:57.927192 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 15230
I20260812 06:55:57.949285 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 15458
I20260812 06:55:57.965978 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:57.950190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.17.69.67:35821)
0812 06:55:57.950795 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:57.950810 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:57.950850 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:55:57.952467 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:57.952477 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:57.952884 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:57.953199 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:57.954626 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.955802 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.957085 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:57.957725 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:57.957889 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:57.960341 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:57.960349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:57.960362 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:57.960409 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:57.962947 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.963606 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.963613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.963619 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.963784 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:57.964149 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:57.964153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:57.964155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:57.964636 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:57.964910 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:57.965306 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 06:55:57.965685 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":298,"threads_started":1}
I20260812 06:55:57.967337 15667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:55:58.049533 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 15458
2026-08-12T06:55:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64445 ms)
[----------] 30 tests from SecurityITest (291799 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:55:58 dist-test-slave-6zbq krb5kdc[15777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:55:58 dist-test-slave-6zbq krb5kdc[15777](info): set up 2 sockets
Aug 12 06:55:58 dist-test-slave-6zbq krb5kdc[15777](info): commencing operation
krb5kdc: starting...
W20260812 06:55:59.306706 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:55:59 dist-test-slave-6zbq krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:55:59.551503 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45393
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:55:59.867213 15800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:55:59.867491 15800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:55:59.867628 15800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:55:59.874497 15800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:55:59.874598 15800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:55:59.874675 15800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:55:59.874763 15800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:55:59.874825 15800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:55:59.883960 15800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:55:59.885061 15800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:55:59.886245 15800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:55:59.896951 15810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:59.897015 15815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:55:59.896992 15809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:55:59.897948 15800 server_base.cc:1034] running on GCE node
I20260812 06:55:59.898653 15800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:55:59.899847 15800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:55:59.901042 15800 hybrid_clock.cc:648] HybridClock initialized: now 1786517759900991 us; error 39 us; skew 500 ppm
I20260812 06:55:59.901351 15800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:55:59 dist-test-slave-6zbq krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:55:59.904271 15800 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:55:59.905545 15800 webserver.cc:460] Webserver started at http://127.17.69.126:43303/ using document root <none> and password file <none>
I20260812 06:55:59.906113 15800 fs_manager.cc:359] Metadata directory not provided
I20260812 06:55:59.906209 15800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:55:59.906495 15800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:55:59.909228 15800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "72c95f5144c6458e8120de79e4cb66cb"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-6zbq"
server_key: "f2879c808041307b8625dfbe3ddfb6a0"
server_key_iv: "ec002e59347d8b1ef39bd29be9f06d00"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.909895 15800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "72c95f5144c6458e8120de79e4cb66cb"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-6zbq"
server_key: "f2879c808041307b8625dfbe3ddfb6a0"
server_key_iv: "ec002e59347d8b1ef39bd29be9f06d00"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.914587 15800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 06:55:59.917629 15832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:55:59.918819 15800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:55:59.918962 15800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "72c95f5144c6458e8120de79e4cb66cb"
format_stamp: "Formatted at 2026-08-12 06:55:59 on dist-test-slave-6zbq"
server_key: "f2879c808041307b8625dfbe3ddfb6a0"
server_key_iv: "ec002e59347d8b1ef39bd29be9f06d00"
server_key_version: "encryptionkey@0"
I20260812 06:55:59.919119 15800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:55:59.935318 15800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:55:59.938766 15800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:55:59.938995 15800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:55:59.955556 15800 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:45393
I20260812 06:55:59.955552 15925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:45393 every 8 connection(s)
I20260812 06:55:59.956844 15800 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:55:59.960316 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:55:59.962725 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 15800
I20260812 06:55:59.962941 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:55:59.963763 17685 external_mini_cluster.cc:1428] Setting key d8adb6aaaa6b1a51ac0ff59417f59c8a
I20260812 06:55:59.973098 15926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: Bootstrap starting.
I20260812 06:55:59.976287 15926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: Neither blocks nor log segments found. Creating new log.
I20260812 06:55:59.977190 15926 log.cc:824] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:55:59 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:55:59.980176 15926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: No bootstrap required, opened a new log
I20260812 06:55:59.986822 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:55:59.965728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.17.69.126:45393)
0812 06:55:59.966480 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0812 06:55:59.966494 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:55:59.966543 (+ 49us) server_negotiation.cc:407] Connection header received
0812 06:55:59.968804 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:55:59.968829 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:55:59.969311 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:55:59.969718 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:55:59.971035 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:59.972892 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:59.974356 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:55:59.975350 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:55:59.975523 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:55:59.979898 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:55:59.979906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:55:59.979919 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:55:59.979956 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:55:59.982912 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:59.983761 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:59.983766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:59.983771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:59.983921 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:55:59.984317 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:55:59.984324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:55:59.984326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:55:59.984834 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:55:59.985049 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 06:55:59.985517 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 06:55:59.985825 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":291,"threads_started":1}
I20260812 06:55:59.988484 15926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } }
I20260812 06:55:59.988900 15926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:55:59.989012 15926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c95f5144c6458e8120de79e4cb66cb, State: Initialized, Role: FOLLOWER
I20260812 06:55:59.989585 15926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } }
I20260812 06:55:59.989828 15926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:55:59.989933 15926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:55:59.990041 15926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:55:59.992084 15926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } }
I20260812 06:55:59.992552 15926 leader_election.cc:302] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c95f5144c6458e8120de79e4cb66cb; no voters:
I20260812 06:55:59.993119 15926 leader_election.cc:288] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:55:59.993204 15935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:55:59.993852 15935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [term 1 LEADER]: Becoming Leader. State: Replica: 72c95f5144c6458e8120de79e4cb66cb, State: Running, Role: LEADER
I20260812 06:55:59.994334 15935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } }
I20260812 06:55:59.994942 15926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:55:59.997028 15938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c95f5144c6458e8120de79e4cb66cb. Latest consensus state: current_term: 1 leader_uuid: "72c95f5144c6458e8120de79e4cb66cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } } }
I20260812 06:55:59.997056 15936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c95f5144c6458e8120de79e4cb66cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c95f5144c6458e8120de79e4cb66cb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45393 } } }
I20260812 06:55:59.997488 15936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:00.002449 15938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:00.003437 15944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:00.008837 15944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:00.009465 15967 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:56:00.017799 15944 catalog_manager.cc:1347] Generated new cluster ID: bf55e4bb80f2449fb62fb80415d2fb8b
I20260812 06:56:00.017901 15944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:00.027081 15944 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:00.028759 15944 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:00.040954 15944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: Generated new TSK 0
I20260812 06:56:00.041831 15944 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:56:00.101837 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:00.415485 15972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:00.415722 15972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:00.415849 15972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:00.423367 15972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:00.423457 15972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:00.423622 15972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:00.432749 15972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:00.433802 15972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:00.435025 15972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:00.446609 15981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.446648 15980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.446667 15989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:00.447499 15972 server_base.cc:1034] running on GCE node
I20260812 06:56:00.448014 15972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:00.448832 15972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:00.450029 15972 hybrid_clock.cc:648] HybridClock initialized: now 1786517760449983 us; error 32 us; skew 500 ppm
I20260812 06:56:00.450325 15972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:00 dist-test-slave-6zbq krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:00.453243 15972 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:56:00.454555 15972 webserver.cc:460] Webserver started at http://127.17.69.65:36539/ using document root <none> and password file <none>
I20260812 06:56:00.455142 15972 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:00.455250 15972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:00.455529 15972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:00.458138 15972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "4de953f10a6845bfa7f0c181b9ff58f4"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "f6fa58c679547bcd3dda4be95f5c532b"
server_key_iv: "c53414e3bd67c13b37902ffdd06c0e68"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.458889 15972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "4de953f10a6845bfa7f0c181b9ff58f4"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "f6fa58c679547bcd3dda4be95f5c532b"
server_key_iv: "c53414e3bd67c13b37902ffdd06c0e68"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.463506 15972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:00.466650 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.467788 15972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 06:56:00.467926 15972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "4de953f10a6845bfa7f0c181b9ff58f4"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "f6fa58c679547bcd3dda4be95f5c532b"
server_key_iv: "c53414e3bd67c13b37902ffdd06c0e68"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.468083 15972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:00.508625 15972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:00.512331 15972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:00.512571 15972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:00.513424 15972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:00.514678 15972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:00.514747 15972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.514855 15972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:00.514906 15972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.544549 15972 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:37431
I20260812 06:56:00.544606 16203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:37431 every 8 connection(s)
I20260812 06:56:00.545665 15972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:00.554226 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 15972
I20260812 06:56:00.554466 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:00.555210 17685 external_mini_cluster.cc:1428] Setting key dcd072ec537e51e717f061c375767901
Aug 12 06:56:00 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:00.563997 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.547879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:50427 (local address 127.17.69.126:45393)
0812 06:56:00.548345 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:00.548352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:00.549196 (+ 844us) server_negotiation.cc:407] Connection header received
0812 06:56:00.550922 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:00.550926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:00.550994 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:00.551086 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:00.552373 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.553068 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.554909 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.555354 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.555454 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.558819 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:00.558824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:00.558828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:00.558859 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:00.561166 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.561827 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.561831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.561833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.561947 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:00.562375 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:00.562380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:00.562383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:00.562654 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:00.562790 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:00.563432 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 06:56:00.563650 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":223,"threads_started":1}
I20260812 06:56:00.564776 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:00.548392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45393 (local address 127.17.69.65:50427)
0812 06:56:00.549044 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:00.549089 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:00.550698 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:00.551265 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:00.551272 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:00.551727 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:00.552234 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.552261 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.553182 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.553185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.554771 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:00.554786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:00.555569 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:00.555576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:00.555844 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:00.556444 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:00.556466 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:00.558637 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:00.561290 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.561299 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.561312 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.561708 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.562048 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:00.562053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:00.562057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:00.562232 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:00.562787 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:00.562794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:00.563069 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 06:56:00.563285 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 06:56:00.563743 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":261,"threads_started":1}
I20260812 06:56:00.566187 16204 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45393
I20260812 06:56:00.566470 16204 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:00.567107 16204 heartbeater.cc:511] Master 127.17.69.126:45393 requested a full tablet report, sending...
I20260812 06:56:00.569151 15861 ts_manager.cc:194] Registered new tserver with Master: 4de953f10a6845bfa7f0c181b9ff58f4 (127.17.69.65:37431)
I20260812 06:56:00.570709 15861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:50427
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:56:00.615119 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:00.931015 16214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:00.931277 16214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:00.931416 16214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:00.938191 16214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:00.938294 16214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:00.938462 16214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:00.947392 16214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:00.948426 16214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:00.949595 16214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:00.961444 16228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.961863 16234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:00.962206 16227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:00.963406 16214 server_base.cc:1034] running on GCE node
I20260812 06:56:00.963919 16214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:00.964753 16214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:00.965965 16214 hybrid_clock.cc:648] HybridClock initialized: now 1786517760965890 us; error 59 us; skew 500 ppm
I20260812 06:56:00.966269 16214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:00 dist-test-slave-6zbq krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:00.969244 16214 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:56:00.970646 16214 webserver.cc:460] Webserver started at http://127.17.69.66:40745/ using document root <none> and password file <none>
I20260812 06:56:00.971246 16214 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:00.971342 16214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:00.971581 16214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:00.974184 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "395a5ad6a726477fbbd7d210d0177d3b"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "21289232592d83f134c6bce6bb99eda3"
server_key_iv: "3ef052df9f36348c99c01d363af51074"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.974905 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "395a5ad6a726477fbbd7d210d0177d3b"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "21289232592d83f134c6bce6bb99eda3"
server_key_iv: "3ef052df9f36348c99c01d363af51074"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.979475 16214 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:00.982582 16244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:00.983731 16214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:00.983848 16214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "395a5ad6a726477fbbd7d210d0177d3b"
format_stamp: "Formatted at 2026-08-12 06:56:00 on dist-test-slave-6zbq"
server_key: "21289232592d83f134c6bce6bb99eda3"
server_key_iv: "3ef052df9f36348c99c01d363af51074"
server_key_version: "encryptionkey@0"
I20260812 06:56:00.983994 16214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:00.996793 16214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:01.000201 16214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:01.000413 16214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:01.001197 16214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:01.002421 16214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:01.002492 16214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.002579 16214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:01.002625 16214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.032990 16214 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:42587
I20260812 06:56:01.033053 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:42587 every 8 connection(s)
I20260812 06:56:01.034152 16214 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:01.037917 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 16214
I20260812 06:56:01.038115 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:01.038589 17685 external_mini_cluster.cc:1428] Setting key 0b02b8187307a9db1eec96cc91b3c789
Aug 12 06:56:01 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517760, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:01.054041 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.036551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:34513 (local address 127.17.69.126:45393)
0812 06:56:01.036720 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.036724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.037851 (+ 1127us) server_negotiation.cc:407] Connection header received
0812 06:56:01.039814 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.039819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.039896 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.039982 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.041499 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.042246 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.044290 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.044685 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.044779 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.048326 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.048332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.048336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.048375 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.051290 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.051949 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.051954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.051956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.052103 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.052461 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.052465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.052468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.052794 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.052925 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.053510 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.053724 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:56:01.054950 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.036986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45393 (local address 127.17.69.66:34513)
0812 06:56:01.037703 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.037745 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.039565 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.040197 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.040205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.040690 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:01.041332 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.041359 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.042438 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.042444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.044161 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.044176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.044895 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.044901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.045143 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.045782 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:01.045808 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:01.048116 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:01.051419 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.051425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.051436 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.051832 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.052188 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.052191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.052193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.052363 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.052896 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:01.052901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:01.053182 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.053377 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.053713 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":268,"threads_started":1}
I20260812 06:56:01.056349 16436 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45393
I20260812 06:56:01.056610 16436 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:01.057255 16436 heartbeater.cc:511] Master 127.17.69.126:45393 requested a full tablet report, sending...
I20260812 06:56:01.058611 15861 ts_manager.cc:194] Registered new tserver with Master: 395a5ad6a726477fbbd7d210d0177d3b (127.17.69.66:42587)
I20260812 06:56:01.059370 15861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:34513
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:56:01.098855 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:01.411453 16447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:01.411715 16447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:01.411868 16447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:01.418735 16447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:01.418838 16447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:01.418977 16447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:01.428210 16447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:01.429283 16447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:01.430474 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:01.442307 16459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:01.442406 16458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:01.442447 16461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:01.443634 16447 server_base.cc:1034] running on GCE node
I20260812 06:56:01.444160 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:01.444968 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:01.446149 16447 hybrid_clock.cc:648] HybridClock initialized: now 1786517761446104 us; error 45 us; skew 500 ppm
I20260812 06:56:01.446475 16447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:01 dist-test-slave-6zbq krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:01.449370 16447 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:56:01.450654 16447 webserver.cc:460] Webserver started at http://127.17.69.67:41491/ using document root <none> and password file <none>
I20260812 06:56:01.451260 16447 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:01.451368 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:01.451614 16447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:01.454187 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "21c0d7b038224685b541b5149f1fca41"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-6zbq"
server_key: "f72238a45d224637892afc94745ac598"
server_key_iv: "9c8fcf7e09fa70a5783df635ec2abbfb"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.454916 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "21c0d7b038224685b541b5149f1fca41"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-6zbq"
server_key: "f72238a45d224637892afc94745ac598"
server_key_iv: "9c8fcf7e09fa70a5783df635ec2abbfb"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.459436 16447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:01.462471 16470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.463598 16447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 06:56:01.463739 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "21c0d7b038224685b541b5149f1fca41"
format_stamp: "Formatted at 2026-08-12 06:56:01 on dist-test-slave-6zbq"
server_key: "f72238a45d224637892afc94745ac598"
server_key_iv: "9c8fcf7e09fa70a5783df635ec2abbfb"
server_key_version: "encryptionkey@0"
I20260812 06:56:01.463913 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:01.497020 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:01.500515 16447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:01.500770 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:01.501600 16447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:01.502776 16447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:01.502844 16447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.502936 16447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:01.502982 16447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:01.530721 16447 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:40081
I20260812 06:56:01.530778 16675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:40081 every 8 connection(s)
I20260812 06:56:01.531853 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:01.541520 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 16447
I20260812 06:56:01.541723 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:01.542204 17685 external_mini_cluster.cc:1428] Setting key dd08128e77086c1da300d6be5e70efb2
Aug 12 06:56:01 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517761, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:01.550607 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.534049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:46457 (local address 127.17.69.126:45393)
0812 06:56:01.534216 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.534220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.535298 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 06:56:01.537109 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.537116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.537194 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.537283 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.538712 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.539461 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.540914 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.541609 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.541769 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.545164 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.545169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.545172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.545204 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.547765 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.548441 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.548445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.548447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.548577 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.548985 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.548988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.548990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.549239 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.549377 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.550033 (+ 656us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.550256 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:56:01.551363 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.534499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:45393 (local address 127.17.69.67:46457)
0812 06:56:01.535149 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.535183 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.536835 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.537513 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.537521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.537989 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:01.538546 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.538565 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.539598 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.539602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.540776 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.540788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.541862 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.541869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.542056 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.542672 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:01.542698 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:01.544978 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:01.547893 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.547902 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.547916 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.548318 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.548680 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:01.548686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:01.548690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:01.548871 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:01.549368 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:01.549375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:01.549682 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.549906 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.550393 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":260,"threads_started":1}
I20260812 06:56:01.552471 16676 heartbeater.cc:347] Connected to a master server at 127.17.69.126:45393
I20260812 06:56:01.552719 16676 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:01.553342 16676 heartbeater.cc:511] Master 127.17.69.126:45393 requested a full tablet report, sending...
I20260812 06:56:01.554616 15861 ts_manager.cc:194] Registered new tserver with Master: 21c0d7b038224685b541b5149f1fca41 (127.17.69.67:40081)
I20260812 06:56:01.555356 15861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:46457
I20260812 06:56:01.557143 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:01.572432 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.561406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.17.69.126:45393)
0812 06:56:01.561535 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.561539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.562007 (+ 468us) server_negotiation.cc:407] Connection header received
0812 06:56:01.562109 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.562112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.562173 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.562243 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.563654 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.564314 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.565460 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.565843 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.565928 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.568062 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.568069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.568073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.568101 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.570223 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.570798 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.570801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.570803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.570921 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.571288 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.571291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.571293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.571565 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.571742 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.572022 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.572169 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:56:01.573699 16204 heartbeater.cc:503] Master 127.17.69.126:45393 was elected leader, sending a full tablet report...
I20260812 06:56:01.575884 15861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:01.582283 15861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:01.613417 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.17.69.66:42587)
0812 06:56:01.602931 (+ 9148us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.602941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.606189 (+ 3248us) server_negotiation.cc:407] Connection header received
0812 06:56:01.606291 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.606299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.606539 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.606698 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:01.607410 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.608556 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.610955 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.612382 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.612506 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.612704 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.612958 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.613120 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8961,"thread_start_us":8840,"threads_started":1}
I20260812 06:56:01.613421 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:42587 (local address 127.0.0.1:49712)
0812 06:56:01.605935 (+ 12595us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.605952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.606069 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.606731 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.606742 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.606772 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:01.607192 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.607205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.608719 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.608725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.610789 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.610802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.612569 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.612575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.612762 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.612798 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.612899 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.613078 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12481,"thread_start_us":13226,"threads_started":1}
I20260812 06:56:01.614711 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40081 (local address 127.0.0.1:52258)
0812 06:56:01.606848 (+ 13844us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.606864 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.606986 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.607629 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.607633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.607671 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:01.608017 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.608029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.609947 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.609953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.612161 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.612178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.613901 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.613908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.614073 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.614105 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.614169 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.614304 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13042,"mutex_wait_us":24,"thread_start_us":11999,"threads_started":1}
I20260812 06:56:01.615547 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:37431 (local address 127.0.0.1:35126)
0812 06:56:01.605240 (+ 12237us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.605298 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.605477 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.608492 (+ 3015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.608498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.608523 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:01.608950 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.608968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.610761 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.610766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.612227 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.612241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.614775 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.614782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.614941 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.614972 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.615064 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.615194 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11986,"mutex_wait_us":24,"thread_start_us":13077,"threads_started":1}
I20260812 06:56:01.615659 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.17.69.65:37431)
0812 06:56:01.607939 (+ 14849us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.607948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.607975 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:56:01.608063 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.608070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.608288 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.608484 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:01.609117 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.610613 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.612428 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.614562 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.614707 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.614934 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.615133 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.615293 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14674,"thread_start_us":14540,"threads_started":1}
I20260812 06:56:01.616083 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.593417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.17.69.67:40081)
0812 06:56:01.606066 (+ 12649us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.606074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.607110 (+ 1036us) server_negotiation.cc:407] Connection header received
0812 06:56:01.607207 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.607214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.607420 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.608208 (+ 788us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:01.608457 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.609802 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.612333 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.613688 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.615322 (+ 1634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.615534 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.615600 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.615748 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12496,"thread_start_us":12823,"threads_started":1}
I20260812 06:56:01.617414 16326 tablet_service.cc:1467] Processing CreateTablet for tablet aa5f873e938d4b90aa2172229becfd03 (DEFAULT_TABLE table=test-table [id=71f9b85a822a42aa8b32386f2c789396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:01.618971 16326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5f873e938d4b90aa2172229becfd03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:01.619169 16555 tablet_service.cc:1467] Processing CreateTablet for tablet aa5f873e938d4b90aa2172229becfd03 (DEFAULT_TABLE table=test-table [id=71f9b85a822a42aa8b32386f2c789396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:01.619181 16078 tablet_service.cc:1467] Processing CreateTablet for tablet aa5f873e938d4b90aa2172229becfd03 (DEFAULT_TABLE table=test-table [id=71f9b85a822a42aa8b32386f2c789396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:01.620235 16555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5f873e938d4b90aa2172229becfd03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:01.620492 16078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5f873e938d4b90aa2172229becfd03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:01.630954 16712 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Bootstrap starting.
I20260812 06:56:01.633765 16715 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Bootstrap starting.
I20260812 06:56:01.634136 16712 tablet_bootstrap.cc:654] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:01.635474 16712 log.cc:824] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:01.637619 16715 tablet_bootstrap.cc:654] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:01.638893 16715 log.cc:824] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:01.639002 16713 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Bootstrap starting.
I20260812 06:56:01.641537 16713 tablet_bootstrap.cc:654] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:01.642697 16713 log.cc:824] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:01.644171 16712 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: No bootstrap required, opened a new log
I20260812 06:56:01.644407 16712 ts_tablet_manager.cc:1397] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260812 06:56:01.644425 16715 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: No bootstrap required, opened a new log
I20260812 06:56:01.644677 16715 ts_tablet_manager.cc:1397] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260812 06:56:01.645579 16713 tablet_bootstrap.cc:492] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: No bootstrap required, opened a new log
I20260812 06:56:01.645829 16713 ts_tablet_manager.cc:1397] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 06:56:01.651403 16712 raft_consensus.cc:348] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.651721 16712 raft_consensus.cc:374] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:01.651826 16712 raft_consensus.cc:729] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c0d7b038224685b541b5149f1fca41, State: Initialized, Role: FOLLOWER
I20260812 06:56:01.652370 16712 consensus_queue.cc:260] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.653352 16676 heartbeater.cc:503] Master 127.17.69.126:45393 was elected leader, sending a full tablet report...
I20260812 06:56:01.653949 16712 ts_tablet_manager.cc:1428] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260812 06:56:01.654604 16715 raft_consensus.cc:348] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.655018 16715 raft_consensus.cc:374] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:01.655110 16715 raft_consensus.cc:729] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de953f10a6845bfa7f0c181b9ff58f4, State: Initialized, Role: FOLLOWER
I20260812 06:56:01.655668 16715 consensus_queue.cc:260] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.655668 16713 raft_consensus.cc:348] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.656090 16713 raft_consensus.cc:374] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:01.656234 16713 raft_consensus.cc:729] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395a5ad6a726477fbbd7d210d0177d3b, State: Initialized, Role: FOLLOWER
I20260812 06:56:01.656883 16713 consensus_queue.cc:260] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.657943 16436 heartbeater.cc:503] Master 127.17.69.126:45393 was elected leader, sending a full tablet report...
I20260812 06:56:01.658566 16713 ts_tablet_manager.cc:1428] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 06:56:01.659551 16715 ts_tablet_manager.cc:1428] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260812 06:56:01.699162 16736 raft_consensus.cc:482] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:01.699386 16736 raft_consensus.cc:504] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.700806 16736 leader_election.cc:288] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4de953f10a6845bfa7f0c181b9ff58f4 (127.17.69.65:37431), 21c0d7b038224685b541b5149f1fca41 (127.17.69.67:40081)
I20260812 06:56:01.708424 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.701192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:52539 (local address 127.17.69.67:40081)
0812 06:56:01.701373 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.701379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.702225 (+ 846us) server_negotiation.cc:407] Connection header received
0812 06:56:01.702406 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.702411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.702468 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.702541 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:01.703192 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.703925 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.706762 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.707501 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.707680 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.707755 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.708056 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.708188 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:56:01.708513 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.701176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:48293 (local address 127.17.69.65:37431)
0812 06:56:01.701374 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.701379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.702173 (+ 794us) server_negotiation.cc:407] Connection header received
0812 06:56:01.702377 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.702383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.702448 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.702520 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:01.703113 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.703867 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.706874 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.707618 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.707768 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.707838 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.708132 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.708264 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:01.708562 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.701222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:40081 (local address 127.17.69.66:52539)
0812 06:56:01.702121 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.702136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.702261 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.702566 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.702570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.702593 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:01.703039 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.703049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.704039 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.704043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.706609 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.706628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.707720 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.707727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.707901 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.707926 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.708011 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.708129 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":135,"spinlock_wait_cycles":40320,"thread_start_us":518,"threads_started":1}
I20260812 06:56:01.708729 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.701192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:37431 (local address 127.17.69.66:48293)
0812 06:56:01.702074 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:01.702089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:01.702223 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:01.702548 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:01.702552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:01.702576 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:01.702976 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.702986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.703992 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.703998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.706688 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:01.706700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.707835 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.707843 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:01.707992 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.708014 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:56:01.708090 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 06:56:01.708206 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":170,"thread_start_us":520,"threads_started":1}
I20260812 06:56:01.709640 16589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5f873e938d4b90aa2172229becfd03" candidate_uuid: "395a5ad6a726477fbbd7d210d0177d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c0d7b038224685b541b5149f1fca41" is_pre_election: true
I20260812 06:56:01.709640 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5f873e938d4b90aa2172229becfd03" candidate_uuid: "395a5ad6a726477fbbd7d210d0177d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" is_pre_election: true
I20260812 06:56:01.710178 16589 raft_consensus.cc:2454] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 395a5ad6a726477fbbd7d210d0177d3b in term 0.
I20260812 06:56:01.710170 16113 raft_consensus.cc:2454] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 395a5ad6a726477fbbd7d210d0177d3b in term 0.
I20260812 06:56:01.710846 16249 leader_election.cc:302] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395a5ad6a726477fbbd7d210d0177d3b, 4de953f10a6845bfa7f0c181b9ff58f4; no voters:
I20260812 06:56:01.711206 16736 raft_consensus.cc:2781] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:01.711341 16736 raft_consensus.cc:482] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:01.711445 16736 raft_consensus.cc:3037] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:01.713536 16736 raft_consensus.cc:504] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.714056 16736 leader_election.cc:288] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [CANDIDATE]: Term 1 election: Requested vote from peers 4de953f10a6845bfa7f0c181b9ff58f4 (127.17.69.65:37431), 21c0d7b038224685b541b5149f1fca41 (127.17.69.67:40081)
I20260812 06:56:01.714509 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5f873e938d4b90aa2172229becfd03" candidate_uuid: "395a5ad6a726477fbbd7d210d0177d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de953f10a6845bfa7f0c181b9ff58f4"
I20260812 06:56:01.714694 16113 raft_consensus.cc:3037] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:01.714691 16589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5f873e938d4b90aa2172229becfd03" candidate_uuid: "395a5ad6a726477fbbd7d210d0177d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c0d7b038224685b541b5149f1fca41"
I20260812 06:56:01.714855 16589 raft_consensus.cc:3037] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:01.716768 16113 raft_consensus.cc:2454] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 395a5ad6a726477fbbd7d210d0177d3b in term 1.
I20260812 06:56:01.716948 16589 raft_consensus.cc:2454] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 395a5ad6a726477fbbd7d210d0177d3b in term 1.
I20260812 06:56:01.717202 16249 leader_election.cc:302] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395a5ad6a726477fbbd7d210d0177d3b, 4de953f10a6845bfa7f0c181b9ff58f4; no voters:
I20260812 06:56:01.717468 16736 raft_consensus.cc:2781] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:01.717933 16736 raft_consensus.cc:686] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 1 LEADER]: Becoming Leader. State: Replica: 395a5ad6a726477fbbd7d210d0177d3b, State: Running, Role: LEADER
I20260812 06:56:01.718432 16736 consensus_queue.cc:237] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } }
I20260812 06:56:01.722399 15861 catalog_manager.cc:5495] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 395a5ad6a726477fbbd7d210d0177d3b (127.17.69.66). New cstate: current_term: 1 leader_uuid: "395a5ad6a726477fbbd7d210d0177d3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "395a5ad6a726477fbbd7d210d0177d3b" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 42587 } health_report { overall_health: HEALTHY } } }
I20260812 06:56:01.740849 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.735657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.17.69.66:42587)
0812 06:56:01.735830 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.735837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.736000 (+ 163us) server_negotiation.cc:407] Connection header received
0812 06:56:01.736202 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.736205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.736273 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.736388 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:01.736766 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.737460 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.738915 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.739307 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.739411 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.739865 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:01.740293 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:01.740414 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.740525 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.740600 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:56:01.747799 16113 raft_consensus.cc:1261] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 1 FOLLOWER]: Refusing update from remote peer 395a5ad6a726477fbbd7d210d0177d3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:01.747810 16589 raft_consensus.cc:1261] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 1 FOLLOWER]: Refusing update from remote peer 395a5ad6a726477fbbd7d210d0177d3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:01.748660 16740 consensus_queue.cc:1035] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c0d7b038224685b541b5149f1fca41" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 40081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:01.748956 16736 consensus_queue.cc:1035] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de953f10a6845bfa7f0c181b9ff58f4" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:01.760557 16744 mvcc.cc:205] Tried to move back new op lower bound from 7317576752110665728 to 7317576752002084864. Current Snapshot: MvccSnapshot[applied={T|T < 7317576752110665728}]
I20260812 06:56:01.771929 15859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:01.772166 15859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:01.775547 15859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c95f5144c6458e8120de79e4cb66cb: Sending DeleteTablet for 3 replicas of tablet aa5f873e938d4b90aa2172229becfd03
I20260812 06:56:01.776588 16078 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5f873e938d4b90aa2172229becfd03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:01 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:35126
I20260812 06:56:01.776707 16555 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5f873e938d4b90aa2172229becfd03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:01 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:52258
I20260812 06:56:01.777323 16757 tablet_replica.cc:330] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: stopping tablet replica
I20260812 06:56:01.777710 16326 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5f873e938d4b90aa2172229becfd03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:01 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:49712
I20260812 06:56:01.778975 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 15972
I20260812 06:56:01.783798 16757 raft_consensus.cc:2229] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:01.784718 16757 raft_consensus.cc:2258] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:01.790788 16761 tablet_replica.cc:330] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: stopping tablet replica
I20260812 06:56:01.791245 16761 raft_consensus.cc:2229] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:01.791484 16757 ts_tablet_manager.cc:1905] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:01.791947 16761 raft_consensus.cc:2258] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:01.793781 16761 ts_tablet_manager.cc:1905] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:01.801016 16761 ts_tablet_manager.cc:1918] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:01.801278 16761 log.cc:1196] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/aa5f873e938d4b90aa2172229becfd03
I20260812 06:56:01.801932 16761 ts_tablet_manager.cc:1939] T aa5f873e938d4b90aa2172229becfd03 P 395a5ad6a726477fbbd7d210d0177d3b: Deleting consensus metadata
I20260812 06:56:01.803668 15836 catalog_manager.cc:4841] TS 395a5ad6a726477fbbd7d210d0177d3b (127.17.69.66:42587): tablet aa5f873e938d4b90aa2172229becfd03 (table test-table [id=71f9b85a822a42aa8b32386f2c789396]) successfully deleted
I20260812 06:56:01.804565 16758 tablet_replica.cc:330] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: stopping tablet replica
I20260812 06:56:01.804971 16758 raft_consensus.cc:2229] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:01.804999 16757 ts_tablet_manager.cc:1918] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:01.805148 16757 log.cc:1196] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/aa5f873e938d4b90aa2172229becfd03
I20260812 06:56:01.805565 16758 raft_consensus.cc:2258] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:01.805665 16757 ts_tablet_manager.cc:1939] T aa5f873e938d4b90aa2172229becfd03 P 4de953f10a6845bfa7f0c181b9ff58f4: Deleting consensus metadata
I20260812 06:56:01.807458 15837 catalog_manager.cc:4841] TS 4de953f10a6845bfa7f0c181b9ff58f4 (127.17.69.65:37431): tablet aa5f873e938d4b90aa2172229becfd03 (table test-table [id=71f9b85a822a42aa8b32386f2c789396]) successfully deleted
I20260812 06:56:01.807730 16758 ts_tablet_manager.cc:1905] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 06:56:01 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:56:01.812781 16758 ts_tablet_manager.cc:1918] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:01.812914 16758 log.cc:1196] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/aa5f873e938d4b90aa2172229becfd03
I20260812 06:56:01.813320 16758 ts_tablet_manager.cc:1939] T aa5f873e938d4b90aa2172229becfd03 P 21c0d7b038224685b541b5149f1fca41: Deleting consensus metadata
I20260812 06:56:01.814806 15833 catalog_manager.cc:4841] TS 21c0d7b038224685b541b5149f1fca41 (127.17.69.67:40081): tablet aa5f873e938d4b90aa2172229becfd03 (table test-table [id=71f9b85a822a42aa8b32386f2c789396]) successfully deleted
I20260812 06:56:01.819027 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.780649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.17.69.65:37431)
0812 06:56:01.780813 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.780819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.796029 (+ 15210us) server_negotiation.cc:407] Connection header received
0812 06:56:01.796448 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.796454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.796539 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.796606 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.798478 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.799611 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.806481 (+ 6870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.807130 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.807230 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.812611 (+ 5381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.812625 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.812641 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.812685 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.816338 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.816967 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.816977 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.816981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.817138 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.817457 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.817462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.817465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.817916 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.818209 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.818433 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.818646 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:56:01.819708 16187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:01.905200 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 15972
I20260812 06:56:01.936528 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 16214
Aug 12 06:56:01 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:56:01.956602 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:01.937449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.17.69.66:42587)
0812 06:56:01.937645 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:01.937651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:01.937669 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:56:01.937730 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:01.937734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:01.937816 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:01.937903 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:01.939251 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.940111 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.941390 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:01.941823 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:01.941932 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:01.950624 (+ 8692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:01.950636 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:01.950652 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:01.950694 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:01.953637 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.954405 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.954411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.954414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.954540 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:01.954919 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:01.954924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:01.954926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:01.955393 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:01.955726 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:01.955995 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 06:56:01.956214 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:56:01.957340 16427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:02.068369 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 16214
I20260812 06:56:02.091289 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 16447
Aug 12 06:56:02 dist-test-slave-6zbq krb5kdc[15777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:56:02.105577 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:02.091985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.17.69.67:40081)
0812 06:56:02.092160 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:02.092165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:02.092186 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:56:02.092394 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:02.092400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:02.092468 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:02.092563 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:02.093740 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:02.094528 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:02.095949 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:02.096381 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:02.096487 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:02.100448 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:02.100462 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:02.100478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:02.100508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:02.102922 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:02.103564 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:02.103571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:02.103573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:02.103739 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:02.104097 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:02.104102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:02.104106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:02.104449 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:02.104709 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:02.105056 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 06:56:02.105263 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:56:02.106174 16666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:02.190631 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 16447
I20260812 06:56:02.208503 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 15800
I20260812 06:56:02.209578 15918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:02.315533 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 15800
2026-08-12T06:56:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4271 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T06:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:02Z Disabled control of system clock
I20260812 06:56:02.401868 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:42065
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:45415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:02.717556 16775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:02.717813 16775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:02.717890 16775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:02.724817 16775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:02.724905 16775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:02.724949 16775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:02.725005 16775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:02.725049 16775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:02.733645 16775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:42065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:02.734732 16775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:02.736339 16775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:02.747166 16786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:02.747249 16789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:02.747619 16795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:02.748191 16775 server_base.cc:1034] running on GCE node
I20260812 06:56:02.748879 16775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:02.750102 16775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:02.751294 16775 hybrid_clock.cc:648] HybridClock initialized: now 1786517762751233 us; error 50 us; skew 500 ppm
I20260812 06:56:02.751600 16775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:02.753610 16775 webserver.cc:460] Webserver started at http://127.17.69.126:45141/ using document root <none> and password file <none>
I20260812 06:56:02.754196 16775 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:02.754302 16775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:02.754623 16775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:02.757296 16775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "a267989a46904a16a7bbb0b31bf72028"
format_stamp: "Formatted at 2026-08-12 06:56:02 on dist-test-slave-6zbq"
server_key: "aa7ac14b69760ae0944c17d43d08dbc9"
server_key_iv: "8a9afb52ac5284c84fa2cb00232e2717"
server_key_version: "encryptionkey@0"
I20260812 06:56:02.758255 16775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "a267989a46904a16a7bbb0b31bf72028"
format_stamp: "Formatted at 2026-08-12 06:56:02 on dist-test-slave-6zbq"
server_key: "aa7ac14b69760ae0944c17d43d08dbc9"
server_key_iv: "8a9afb52ac5284c84fa2cb00232e2717"
server_key_version: "encryptionkey@0"
I20260812 06:56:02.762862 16775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:56:02.765990 16807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:02.767243 16775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:02.767403 16775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "a267989a46904a16a7bbb0b31bf72028"
format_stamp: "Formatted at 2026-08-12 06:56:02 on dist-test-slave-6zbq"
server_key: "aa7ac14b69760ae0944c17d43d08dbc9"
server_key_iv: "8a9afb52ac5284c84fa2cb00232e2717"
server_key_version: "encryptionkey@0"
I20260812 06:56:02.767591 16775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:02.790676 16775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:02.791379 16775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:02.791625 16775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:02.808354 16775 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:42065
I20260812 06:56:02.808411 16887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:42065 every 8 connection(s)
I20260812 06:56:02.809589 16775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:02.812288 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 16775
I20260812 06:56:02.812456 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:02.813089 17685 external_mini_cluster.cc:1428] Setting key 8050eb61435c20cabe663dfe1722f1e3
I20260812 06:56:02.813573 16890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:02.827630 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:02.814915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.17.69.126:42065)
0812 06:56:02.815950 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:02.815964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:02.816019 (+ 55us) server_negotiation.cc:407] Connection header received
0812 06:56:02.818161 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:02.818190 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:02.818754 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:02.819150 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:02.819676 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:02.821657 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:02.822963 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:02.823928 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:02.824108 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:02.825091 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:02.825100 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:02.825116 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:02.825809 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:02.825899 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:02.825923 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:02.826024 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:02.826157 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 06:56:02.826460 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":345,"threads_started":1}
I20260812 06:56:02.827885 16890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: Bootstrap starting.
I20260812 06:56:02.830814 16890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:02.831714 16890 log.cc:824] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:02.834245 16890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: No bootstrap required, opened a new log
I20260812 06:56:02.841003 16890 raft_consensus.cc:348] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } }
I20260812 06:56:02.841331 16890 raft_consensus.cc:374] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:02.841432 16890 raft_consensus.cc:729] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a267989a46904a16a7bbb0b31bf72028, State: Initialized, Role: FOLLOWER
I20260812 06:56:02.841935 16890 consensus_queue.cc:260] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } }
I20260812 06:56:02.842190 16890 raft_consensus.cc:388] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:02.842303 16890 raft_consensus.cc:482] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:02.842444 16890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:02.844300 16890 raft_consensus.cc:504] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } }
I20260812 06:56:02.844686 16890 leader_election.cc:302] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a267989a46904a16a7bbb0b31bf72028; no voters:
I20260812 06:56:02.845148 16890 leader_election.cc:288] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:02.845268 16896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:02.845911 16896 raft_consensus.cc:686] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [term 1 LEADER]: Becoming Leader. State: Replica: a267989a46904a16a7bbb0b31bf72028, State: Running, Role: LEADER
I20260812 06:56:02.846424 16896 consensus_queue.cc:237] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } }
I20260812 06:56:02.847177 16890 sys_catalog.cc:564] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:02.849195 16899 sys_catalog.cc:455] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a267989a46904a16a7bbb0b31bf72028. Latest consensus state: current_term: 1 leader_uuid: "a267989a46904a16a7bbb0b31bf72028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } } }
I20260812 06:56:02.849675 16899 sys_catalog.cc:458] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:02.849337 16897 sys_catalog.cc:455] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a267989a46904a16a7bbb0b31bf72028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267989a46904a16a7bbb0b31bf72028" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 42065 } } }
I20260812 06:56:02.850979 16897 sys_catalog.cc:458] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:02.852289 16907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:02.857302 16907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:02.868150 16907 catalog_manager.cc:1347] Generated new cluster ID: 2882d2c7beda448cac1cf2f993667970
I20260812 06:56:02.868259 16907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:02.887750 16907 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:02.889400 16907 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:02.899086 16907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: Generated new TSK 0
I20260812 06:56:02.900110 16907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:02.906898 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--builtin_ntp_servers=127.17.69.84:45415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:03.252326 16933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:03.252604 16933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:03.252728 16933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:03.259658 16933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:03.259747 16933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:03.259886 16933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:03.268394 16933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:03.269388 16933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:03.271147 16933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:03.282716 16947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:03.282750 16945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:03.282671 16944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:03.283560 16933 server_base.cc:1034] running on GCE node
I20260812 06:56:03.284061 16933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:03.284845 16933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:03.286036 16933 hybrid_clock.cc:648] HybridClock initialized: now 1786517763285984 us; error 42 us; skew 500 ppm
I20260812 06:56:03.286301 16933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:03.288326 16933 webserver.cc:460] Webserver started at http://127.17.69.65:42931/ using document root <none> and password file <none>
I20260812 06:56:03.288913 16933 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:03.289005 16933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:03.289285 16933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:03.291846 16933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "37c7060d418644beacc00dc40b24e856"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "0d354fb7dc2cfc7f475ec24603fe5188"
server_key_iv: "e25a7dccdd5e9557309d3441422154cb"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.292554 16933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "37c7060d418644beacc00dc40b24e856"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "0d354fb7dc2cfc7f475ec24603fe5188"
server_key_iv: "e25a7dccdd5e9557309d3441422154cb"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.297011 16933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 06:56:03.300063 16963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.301208 16933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 06:56:03.301342 16933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "37c7060d418644beacc00dc40b24e856"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "0d354fb7dc2cfc7f475ec24603fe5188"
server_key_iv: "e25a7dccdd5e9557309d3441422154cb"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.301522 16933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:03.321381 16933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:03.322041 16933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:03.322297 16933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:03.323158 16933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:03.324433 16933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:03.324527 16933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.324633 16933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:03.324705 16933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.354480 16933 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:33569
I20260812 06:56:03.354568 17171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:33569 every 8 connection(s)
I20260812 06:56:03.355633 16933 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:03.361408 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 16933
I20260812 06:56:03.361589 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:03.362010 17685 external_mini_cluster.cc:1428] Setting key 271f659df606d6556d74e86c29d47ba2
I20260812 06:56:03.364724 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--builtin_ntp_servers=127.17.69.84:45415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:03.367697 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:03.357906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:36693 (local address 127.17.69.126:42065)
0812 06:56:03.358374 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:03.358381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:03.359472 (+ 1091us) server_negotiation.cc:407] Connection header received
0812 06:56:03.361143 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:03.361149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:03.361236 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:03.361357 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:03.362417 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.363286 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.364870 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.365401 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.365525 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:03.366855 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:03.366862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:03.366866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:03.366888 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:03.366935 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:03.366950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:03.367038 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:03.367285 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 06:56:03.367383 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":210,"threads_started":1}
I20260812 06:56:03.368522 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:03.358491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:42065 (local address 127.17.69.65:36693)
0812 06:56:03.359321 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:03.359360 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:03.360880 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:03.361516 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:03.361526 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:03.361629 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:03.362225 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:03.362244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.363439 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.363443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:03.364720 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:03.364734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.365592 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.365598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:03.365825 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:03.366535 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:03.366538 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:03.366576 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:03.366663 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:03.367072 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:03.367083 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:03.367089 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:03.367258 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 06:56:03.367540 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":251,"threads_started":1}
I20260812 06:56:03.369818 17173 heartbeater.cc:347] Connected to a master server at 127.17.69.126:42065
I20260812 06:56:03.370119 17173 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:03.371011 17173 heartbeater.cc:511] Master 127.17.69.126:42065 requested a full tablet report, sending...
I20260812 06:56:03.373004 16834 ts_manager.cc:194] Registered new tserver with Master: 37c7060d418644beacc00dc40b24e856 (127.17.69.65:33569)
I20260812 06:56:03.374434 16834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.65:36693
W20260812 06:56:03.685644 17180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:03.685871 17180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:03.685976 17180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:03.692809 17180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:03.692891 17180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:03.693027 17180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:03.701419 17180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:03.702468 17180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:03.704052 17180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:03.715925 17194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:03.715924 17192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:03.716130 17191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:03.716526 17180 server_base.cc:1034] running on GCE node
I20260812 06:56:03.717088 17180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:03.717891 17180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:03.719089 17180 hybrid_clock.cc:648] HybridClock initialized: now 1786517763719046 us; error 30 us; skew 500 ppm
I20260812 06:56:03.719379 17180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:03.721441 17180 webserver.cc:460] Webserver started at http://127.17.69.66:43643/ using document root <none> and password file <none>
I20260812 06:56:03.722065 17180 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:03.722162 17180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:03.722478 17180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:03.725005 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "24ed5e6ea01e4898b45c0f690a8f4175"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "c18089649e231c8d5355e3c452bca58d"
server_key_iv: "5717fc1f9d3ede2e4a72936bbb949af2"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.725713 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "24ed5e6ea01e4898b45c0f690a8f4175"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "c18089649e231c8d5355e3c452bca58d"
server_key_iv: "5717fc1f9d3ede2e4a72936bbb949af2"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.730197 17180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 06:56:03.733306 17207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.734489 17180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:03.734632 17180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "24ed5e6ea01e4898b45c0f690a8f4175"
format_stamp: "Formatted at 2026-08-12 06:56:03 on dist-test-slave-6zbq"
server_key: "c18089649e231c8d5355e3c452bca58d"
server_key_iv: "5717fc1f9d3ede2e4a72936bbb949af2"
server_key_version: "encryptionkey@0"
I20260812 06:56:03.734822 17180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:03.756748 17180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:03.757401 17180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:03.757651 17180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:03.758495 17180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:03.759749 17180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:03.759846 17180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.759940 17180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:03.760006 17180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:03.788720 17180 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:43943
I20260812 06:56:03.788797 17404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:43943 every 8 connection(s)
I20260812 06:56:03.789846 17180 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:03.793069 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17180
I20260812 06:56:03.793233 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:03.793682 17685 external_mini_cluster.cc:1428] Setting key ebaaa34eb40936a7797fc9ee78968fa7
I20260812 06:56:03.796304 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--builtin_ntp_servers=127.17.69.84:45415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:03.801848 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:03.792108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:37987 (local address 127.17.69.126:42065)
0812 06:56:03.792273 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:03.792277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:03.793722 (+ 1445us) server_negotiation.cc:407] Connection header received
0812 06:56:03.795455 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:03.795461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:03.795534 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:03.795618 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:03.796655 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.797375 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.799274 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.799774 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.799889 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:03.801106 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:03.801117 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:03.801120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:03.801137 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:03.801167 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:03.801178 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:03.801251 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:03.801452 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 06:56:03.801549 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:56:03.802773 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:03.792585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:42065 (local address 127.17.69.66:37987)
0812 06:56:03.793547 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:03.793597 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:03.795214 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:03.795844 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:03.795852 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:03.795937 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:03.796501 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:03.796522 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.797530 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.797534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:03.799137 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:03.799150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:03.799987 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:03.799995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:03.800209 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:03.800810 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:03.800812 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:03.800847 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:03.800918 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:03.801266 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:03.801277 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:03.801285 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 06:56:03.801427 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 06:56:03.801787 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":300,"threads_started":1}
I20260812 06:56:03.804018 17407 heartbeater.cc:347] Connected to a master server at 127.17.69.126:42065
I20260812 06:56:03.804279 17407 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:03.804930 17407 heartbeater.cc:511] Master 127.17.69.126:42065 requested a full tablet report, sending...
I20260812 06:56:03.806234 16834 ts_manager.cc:194] Registered new tserver with Master: 24ed5e6ea01e4898b45c0f690a8f4175 (127.17.69.66:43943)
I20260812 06:56:03.807008 16834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.66:37987
W20260812 06:56:04.114614 17411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:04.114850 17411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:04.114966 17411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:04.121830 17411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:04.121918 17411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:04.122056 17411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:04.130576 17411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:04.131675 17411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:04.133337 17411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:04.145314 17420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:04.145373 17427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:04.145357 17423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:04.146428 17411 server_base.cc:1034] running on GCE node
I20260812 06:56:04.146924 17411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:04.147778 17411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:04.148965 17411 hybrid_clock.cc:648] HybridClock initialized: now 1786517764148934 us; error 27 us; skew 500 ppm
I20260812 06:56:04.149240 17411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:04.151227 17411 webserver.cc:460] Webserver started at http://127.17.69.67:33865/ using document root <none> and password file <none>
I20260812 06:56:04.151840 17411 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:04.151932 17411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:04.152225 17411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:04.154930 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "29168d182e00475ea87bc8d35ef62bb1"
format_stamp: "Formatted at 2026-08-12 06:56:04 on dist-test-slave-6zbq"
server_key: "7438f82f6ef978f4c78a2e39b47a3e3a"
server_key_iv: "0d36455d2c7b19ee3c1813b7dd29a63c"
server_key_version: "encryptionkey@0"
I20260812 06:56:04.155661 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "29168d182e00475ea87bc8d35ef62bb1"
format_stamp: "Formatted at 2026-08-12 06:56:04 on dist-test-slave-6zbq"
server_key: "7438f82f6ef978f4c78a2e39b47a3e3a"
server_key_iv: "0d36455d2c7b19ee3c1813b7dd29a63c"
server_key_version: "encryptionkey@0"
I20260812 06:56:04.160207 17411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:04.163278 17441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:04.164417 17411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:04.164544 17411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "29168d182e00475ea87bc8d35ef62bb1"
format_stamp: "Formatted at 2026-08-12 06:56:04 on dist-test-slave-6zbq"
server_key: "7438f82f6ef978f4c78a2e39b47a3e3a"
server_key_iv: "0d36455d2c7b19ee3c1813b7dd29a63c"
server_key_version: "encryptionkey@0"
I20260812 06:56:04.164721 17411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:04.183753 17411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:04.184414 17411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:04.184664 17411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:04.185490 17411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:04.186798 17411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:04.186884 17411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:04.186982 17411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:04.187057 17411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:04.216230 17411 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:35391
I20260812 06:56:04.216302 17656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:35391 every 8 connection(s)
I20260812 06:56:04.217373 17411 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:04.225677 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17411
I20260812 06:56:04.225832 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:04.226250 17685 external_mini_cluster.cc:1428] Setting key 5e12d20544d352deeda004139e501410
I20260812 06:56:04.228701 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.219616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:60709 (local address 127.17.69.126:42065)
0812 06:56:04.219788 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.219792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.221013 (+ 1221us) server_negotiation.cc:407] Connection header received
0812 06:56:04.222726 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.222732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.222814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.222903 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:04.223857 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.224594 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.226126 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.226575 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.226689 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.227912 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:04.227917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:04.227921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:04.227944 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:04.227984 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:04.228000 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:04.228091 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.228291 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.228385 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:56:04.229465 17662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.220051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:42065 (local address 127.17.69.67:60709)
0812 06:56:04.220866 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.220906 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.222475 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.223112 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.223119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.223197 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:04.223691 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.223706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.224752 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.224756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.225988 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.226000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.226790 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.226799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.226995 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.227625 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:04.227628 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:04.227661 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:04.227733 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:04.228098 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:04.228109 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:04.228115 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.228251 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.228499 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":248,"threads_started":1}
I20260812 06:56:04.230654 17658 heartbeater.cc:347] Connected to a master server at 127.17.69.126:42065
I20260812 06:56:04.230901 17658 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:04.231544 17658 heartbeater.cc:511] Master 127.17.69.126:42065 requested a full tablet report, sending...
I20260812 06:56:04.232766 16834 ts_manager.cc:194] Registered new tserver with Master: 29168d182e00475ea87bc8d35ef62bb1 (127.17.69.67:35391)
I20260812 06:56:04.233511 16834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.67:60709
I20260812 06:56:04.240701 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:04.250032 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.244524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.17.69.126:42065)
0812 06:56:04.244658 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.244663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.245087 (+ 424us) server_negotiation.cc:407] Connection header received
0812 06:56:04.245232 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.245235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.245311 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.245386 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:04.245844 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.246512 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.247928 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.248312 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.248421 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.249410 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:04.249415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:04.249418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:04.249434 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:04.249464 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:04.249475 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:04.249545 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.249680 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.249771 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:56:04.253296 16834 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:04.259675 16834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:04.288420 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.270065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:43943 (local address 127.0.0.1:53430)
0812 06:56:04.271022 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.271065 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.271216 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.281279 (+ 10063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.281286 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.281315 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:04.281703 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.281717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.283547 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.283552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.285831 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.285847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.287542 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.287549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.287726 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.287788 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.287908 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.288076 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":33,"thread_start_us":11438,"threads_started":1}
I20260812 06:56:04.288522 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.270372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.17.69.66:43943)
0812 06:56:04.280700 (+ 10328us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.280708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.280739 (+ 31us) server_negotiation.cc:407] Connection header received
0812 06:56:04.280834 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.280840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.281063 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.281267 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:04.281889 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.283394 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.285981 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.287352 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.287480 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.287909 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.287984 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.288158 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10165,"thread_start_us":10044,"threads_started":1}
I20260812 06:56:04.291481 17284 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a (DEFAULT_TABLE table=test-table [id=64252e4fff2846bea45485b328ca28a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:04.292675 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.270441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:33569 (local address 127.0.0.1:34120)
0812 06:56:04.280936 (+ 10495us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.280952 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.281055 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.283797 (+ 2742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.283801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.283831 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:04.286025 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.286034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.287756 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.287761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.290522 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.290535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.291724 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.291731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.291901 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.291933 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.292193 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.292317 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10392,"mutex_wait_us":1754,"thread_start_us":683,"threads_started":1}
I20260812 06:56:04.292722 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.270927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.17.69.65:33569)
0812 06:56:04.283283 (+ 12356us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.283290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.283314 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:56:04.283396 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.283402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.283611 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.283793 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:04.286164 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.287591 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.290686 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.291549 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.291667 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.291884 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.292260 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.292415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12216,"thread_start_us":12115,"threads_started":1}
I20260812 06:56:04.292842 17284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6967e4a3e4f06a600d592fde9e51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:04.295431 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.271778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:35391 (local address 127.0.0.1:39524)
0812 06:56:04.282738 (+ 10960us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.282754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.282855 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.284120 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.284128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.284160 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:04.284580 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.284591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.286328 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.286333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.287656 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.287670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.294671 (+ 7001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.294679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.294865 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.294898 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.294990 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.295116 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10848,"mutex_wait_us":3,"thread_start_us":10726,"threads_started":1}
I20260812 06:56:04.295557 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.272072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.17.69.67:35391)
0812 06:56:04.283626 (+ 11554us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.283631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.283654 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:56:04.283716 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.283720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.283918 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.284109 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:04.284816 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.286212 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.287881 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.289156 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.289299 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.289549 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.295052 (+ 5503us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.295210 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11436,"thread_start_us":11307,"threads_started":1}
I20260812 06:56:04.295675 17056 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a (DEFAULT_TABLE table=test-table [id=64252e4fff2846bea45485b328ca28a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:04.296802 17056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6967e4a3e4f06a600d592fde9e51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:04.299311 17522 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a (DEFAULT_TABLE table=test-table [id=64252e4fff2846bea45485b328ca28a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:04.300625 17522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee6967e4a3e4f06a600d592fde9e51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:04.304504 17697 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Bootstrap starting.
I20260812 06:56:04.307377 17697 tablet_bootstrap.cc:654] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:04.308286 17697 log.cc:824] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:04.311125 17697 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: No bootstrap required, opened a new log
I20260812 06:56:04.311398 17697 ts_tablet_manager.cc:1397] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260812 06:56:04.311741 17700 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Bootstrap starting.
I20260812 06:56:04.314209 17701 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Bootstrap starting.
I20260812 06:56:04.315220 17700 tablet_bootstrap.cc:654] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:04.316345 17700 log.cc:824] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:04.316872 17701 tablet_bootstrap.cc:654] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:04.317925 17701 log.cc:824] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:04.319492 17700 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: No bootstrap required, opened a new log
I20260812 06:56:04.319748 17700 ts_tablet_manager.cc:1397] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260812 06:56:04.320437 17701 tablet_bootstrap.cc:492] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: No bootstrap required, opened a new log
I20260812 06:56:04.320655 17701 ts_tablet_manager.cc:1397] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 06:56:04.320781 17697 raft_consensus.cc:348] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.321223 17697 raft_consensus.cc:374] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:04.321321 17697 raft_consensus.cc:729] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ed5e6ea01e4898b45c0f690a8f4175, State: Initialized, Role: FOLLOWER
I20260812 06:56:04.321884 17697 consensus_queue.cc:260] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.322953 17407 heartbeater.cc:503] Master 127.17.69.126:42065 was elected leader, sending a full tablet report...
I20260812 06:56:04.324744 17697 ts_tablet_manager.cc:1428] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Time spent starting tablet: real 0.013s user 0.002s sys 0.010s
I20260812 06:56:04.328197 17700 raft_consensus.cc:348] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.328513 17700 raft_consensus.cc:374] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:04.328626 17700 raft_consensus.cc:729] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c7060d418644beacc00dc40b24e856, State: Initialized, Role: FOLLOWER
I20260812 06:56:04.328848 17701 raft_consensus.cc:348] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.329252 17701 raft_consensus.cc:374] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:04.329139 17700 consensus_queue.cc:260] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.329345 17701 raft_consensus.cc:729] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29168d182e00475ea87bc8d35ef62bb1, State: Initialized, Role: FOLLOWER
I20260812 06:56:04.329852 17701 consensus_queue.cc:260] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.330263 17173 heartbeater.cc:503] Master 127.17.69.126:42065 was elected leader, sending a full tablet report...
I20260812 06:56:04.330824 17700 ts_tablet_manager.cc:1428] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260812 06:56:04.331228 17658 heartbeater.cc:503] Master 127.17.69.126:42065 was elected leader, sending a full tablet report...
I20260812 06:56:04.331492 17701 ts_tablet_manager.cc:1428] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Time spent starting tablet: real 0.011s user 0.004s sys 0.007s
W20260812 06:56:04.357576 17176 tablet.cc:2369] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:04.468856 17661 tablet.cc:2369] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:04.486694 17707 raft_consensus.cc:482] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:04.486986 17707 raft_consensus.cc:504] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.488788 17707 leader_election.cc:288] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24ed5e6ea01e4898b45c0f690a8f4175 (127.17.69.66:43943), 29168d182e00475ea87bc8d35ef62bb1 (127.17.69.67:35391)
I20260812 06:56:04.495157 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.489243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:47947 (local address 127.17.69.66:43943)
0812 06:56:04.489436 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.489441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.490141 (+ 700us) server_negotiation.cc:407] Connection header received
0812 06:56:04.490288 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.490292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.490385 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.490461 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:04.490988 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.491666 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.493546 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.494305 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.494470 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.494537 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.494808 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.494929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:56:04.495265 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.489063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:43943 (local address 127.17.69.65:47947)
0812 06:56:04.489967 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.489991 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.490161 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.490489 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.490493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.490515 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:04.490833 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.490844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.491804 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.491809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.493352 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.493364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.494491 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.494498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.494647 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.494675 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.494753 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.494874 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":76,"spinlock_wait_cycles":13696,"thread_start_us":557,"threads_started":1}
I20260812 06:56:04.495328 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.489236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:58725 (local address 127.17.69.67:35391)
0812 06:56:04.489429 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.489435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.490086 (+ 651us) server_negotiation.cc:407] Connection header received
0812 06:56:04.490627 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.490631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.490687 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.490750 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:04.491336 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.492037 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.493701 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.494501 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.494652 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.494718 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.494978 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.495100 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:04.495725 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.489193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:35391 (local address 127.17.69.65:58725)
0812 06:56:04.489984 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:04.489997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:04.490161 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:04.490805 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:04.490809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:04.490836 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:04.491168 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.491177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.492169 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.492176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.493523 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:04.493536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.494680 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.494686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:04.494833 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.494856 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 06:56:04.494931 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 06:56:04.495044 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"spinlock_wait_cycles":14464,"thread_start_us":420,"threads_started":1}
I20260812 06:56:04.496322 17318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee6967e4a3e4f06a600d592fde9e51a" candidate_uuid: "37c7060d418644beacc00dc40b24e856" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" is_pre_election: true
I20260812 06:56:04.496654 17558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee6967e4a3e4f06a600d592fde9e51a" candidate_uuid: "37c7060d418644beacc00dc40b24e856" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29168d182e00475ea87bc8d35ef62bb1" is_pre_election: true
I20260812 06:56:04.496791 17318 raft_consensus.cc:2454] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37c7060d418644beacc00dc40b24e856 in term 0.
I20260812 06:56:04.497069 17558 raft_consensus.cc:2454] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37c7060d418644beacc00dc40b24e856 in term 0.
I20260812 06:56:04.497426 16964 leader_election.cc:302] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ed5e6ea01e4898b45c0f690a8f4175, 37c7060d418644beacc00dc40b24e856; no voters:
I20260812 06:56:04.497826 17707 raft_consensus.cc:2781] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:04.497968 17707 raft_consensus.cc:482] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:04.498055 17707 raft_consensus.cc:3037] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:04.500177 17707 raft_consensus.cc:504] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.500715 17707 leader_election.cc:288] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [CANDIDATE]: Term 1 election: Requested vote from peers 24ed5e6ea01e4898b45c0f690a8f4175 (127.17.69.66:43943), 29168d182e00475ea87bc8d35ef62bb1 (127.17.69.67:35391)
I20260812 06:56:04.501127 17318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee6967e4a3e4f06a600d592fde9e51a" candidate_uuid: "37c7060d418644beacc00dc40b24e856" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ed5e6ea01e4898b45c0f690a8f4175"
I20260812 06:56:04.501179 17558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee6967e4a3e4f06a600d592fde9e51a" candidate_uuid: "37c7060d418644beacc00dc40b24e856" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29168d182e00475ea87bc8d35ef62bb1"
I20260812 06:56:04.501291 17318 raft_consensus.cc:3037] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:04.501360 17558 raft_consensus.cc:3037] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:04.503477 17558 raft_consensus.cc:2454] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37c7060d418644beacc00dc40b24e856 in term 1.
I20260812 06:56:04.503485 17318 raft_consensus.cc:2454] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37c7060d418644beacc00dc40b24e856 in term 1.
I20260812 06:56:04.503824 16964 leader_election.cc:302] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ed5e6ea01e4898b45c0f690a8f4175, 37c7060d418644beacc00dc40b24e856; no voters:
I20260812 06:56:04.504102 17707 raft_consensus.cc:2781] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:04.504489 17707 raft_consensus.cc:686] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 1 LEADER]: Becoming Leader. State: Replica: 37c7060d418644beacc00dc40b24e856, State: Running, Role: LEADER
I20260812 06:56:04.504920 17707 consensus_queue.cc:237] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } }
I20260812 06:56:04.509240 16834 catalog_manager.cc:5495] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37c7060d418644beacc00dc40b24e856 (127.17.69.65). New cstate: current_term: 1 leader_uuid: "37c7060d418644beacc00dc40b24e856" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37c7060d418644beacc00dc40b24e856" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:04.523399 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.518740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.17.69.65:33569)
0812 06:56:04.518905 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.518910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.518925 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:56:04.518991 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.518995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.519068 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.519184 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:04.519506 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.520150 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.521567 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.521955 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.522039 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.522520 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:04.522952 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:04.523062 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.523104 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.523163 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:56:04.530465 17558 raft_consensus.cc:1261] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 1 FOLLOWER]: Refusing update from remote peer 37c7060d418644beacc00dc40b24e856: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:04.530465 17318 raft_consensus.cc:1261] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 1 FOLLOWER]: Refusing update from remote peer 37c7060d418644beacc00dc40b24e856: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:04.531404 17716 consensus_queue.cc:1035] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29168d182e00475ea87bc8d35ef62bb1" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 35391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:04.531754 17707 consensus_queue.cc:1035] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24ed5e6ea01e4898b45c0f690a8f4175" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:56:04.542085 17408 tablet.cc:2369] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:04.542866 17720 mvcc.cc:205] Tried to move back new op lower bound from 7317576763508125696 to 7317576763415388160. Current Snapshot: MvccSnapshot[applied={T|T < 7317576763508125696}]
I20260812 06:56:04.547585 17726 mvcc.cc:205] Tried to move back new op lower bound from 7317576763508125696 to 7317576763415388160. Current Snapshot: MvccSnapshot[applied={T|T < 7317576763508125696}]
I20260812 06:56:04.555642 16834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:04.555861 16834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:04.559186 16834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a267989a46904a16a7bbb0b31bf72028: Sending DeleteTablet for 3 replicas of tablet 8ee6967e4a3e4f06a600d592fde9e51a
I20260812 06:56:04.560209 17284 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:04 UTC) from {username='slave'} at 127.0.0.1:53430
I20260812 06:56:04.560503 17056 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:04 UTC) from {username='slave'} at 127.0.0.1:34120
I20260812 06:56:04.561197 17522 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee6967e4a3e4f06a600d592fde9e51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:04 UTC) from {username='slave'} at 127.0.0.1:39524
I20260812 06:56:04.562119 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 16933
I20260812 06:56:04.568267 17739 tablet_replica.cc:330] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: stopping tablet replica
I20260812 06:56:04.568665 17739 raft_consensus.cc:2229] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:04.569002 17740 tablet_replica.cc:330] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: stopping tablet replica
I20260812 06:56:04.569392 17739 raft_consensus.cc:2258] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:04.569407 17740 raft_consensus.cc:2229] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:04.569875 17740 raft_consensus.cc:2258] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:04.570631 17736 tablet_replica.cc:330] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: stopping tablet replica
I20260812 06:56:04.570974 17736 raft_consensus.cc:2229] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:04.571419 17736 raft_consensus.cc:2258] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:04.571730 17739 ts_tablet_manager.cc:1905] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:04.573243 17740 ts_tablet_manager.cc:1905] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:04.573437 17736 ts_tablet_manager.cc:1905] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:04.578774 17739 ts_tablet_manager.cc:1918] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:04.578917 17739 log.cc:1196] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/8ee6967e4a3e4f06a600d592fde9e51a
I20260812 06:56:04.579449 17739 ts_tablet_manager.cc:1939] T 8ee6967e4a3e4f06a600d592fde9e51a P 37c7060d418644beacc00dc40b24e856: Deleting consensus metadata
I20260812 06:56:04.579505 17740 ts_tablet_manager.cc:1918] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:04.579651 17740 log.cc:1196] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/8ee6967e4a3e4f06a600d592fde9e51a
I20260812 06:56:04.580112 17740 ts_tablet_manager.cc:1939] T 8ee6967e4a3e4f06a600d592fde9e51a P 29168d182e00475ea87bc8d35ef62bb1: Deleting consensus metadata
I20260812 06:56:04.580427 17736 ts_tablet_manager.cc:1918] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:04.580619 17736 log.cc:1196] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/8ee6967e4a3e4f06a600d592fde9e51a
I20260812 06:56:04.581192 17736 ts_tablet_manager.cc:1939] T 8ee6967e4a3e4f06a600d592fde9e51a P 24ed5e6ea01e4898b45c0f690a8f4175: Deleting consensus metadata
I20260812 06:56:04.581139 16811 catalog_manager.cc:4841] TS 37c7060d418644beacc00dc40b24e856 (127.17.69.65:33569): tablet 8ee6967e4a3e4f06a600d592fde9e51a (table test-table [id=64252e4fff2846bea45485b328ca28a1]) successfully deleted
I20260812 06:56:04.581601 16810 catalog_manager.cc:4841] TS 29168d182e00475ea87bc8d35ef62bb1 (127.17.69.67:35391): tablet 8ee6967e4a3e4f06a600d592fde9e51a (table test-table [id=64252e4fff2846bea45485b328ca28a1]) successfully deleted
I20260812 06:56:04.582887 16808 catalog_manager.cc:4841] TS 24ed5e6ea01e4898b45c0f690a8f4175 (127.17.69.66:43943): tablet 8ee6967e4a3e4f06a600d592fde9e51a (table test-table [id=64252e4fff2846bea45485b328ca28a1]) successfully deleted
I20260812 06:56:04.587193 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.563405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.17.69.65:33569)
0812 06:56:04.565354 (+ 1949us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.565361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.568454 (+ 3093us) server_negotiation.cc:407] Connection header received
0812 06:56:04.568617 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.568622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.568718 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.568812 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:04.581768 (+ 12956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.582588 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.584027 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.584663 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.584801 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.585723 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:04.585738 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:04.585753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:04.586480 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:04.586534 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:04.586558 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:04.586675 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.586743 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.586842 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1815}
I20260812 06:56:04.587860 17164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:04.686842 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 16933
I20260812 06:56:04.710039 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17180
I20260812 06:56:04.716420 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.710736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.17.69.66:43943)
0812 06:56:04.710891 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.710896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.710917 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:56:04.711069 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.711072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.711185 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.711271 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:04.711687 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.712449 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.713754 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.714124 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.714215 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.715318 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:04.715327 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:04.715340 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:04.715883 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:04.715929 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:04.715948 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:04.716030 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.716099 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.716181 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:56:04.717020 17395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:04.802021 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17180
I20260812 06:56:04.827103 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17411
I20260812 06:56:04.833129 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:04.827831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.17.69.67:35391)
0812 06:56:04.827995 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:04.828000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:04.828017 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:56:04.828131 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:04.828135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:04.828217 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:04.828294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:04.828616 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.829350 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.830576 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:04.830970 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:04.831058 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:04.831891 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:04.831900 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:04.831915 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:04.832468 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:04.832515 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:04.832534 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:04.832615 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:04.832745 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 06:56:04.832837 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:56:04.833714 17647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:04.917629 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17411
I20260812 06:56:04.939693 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 16775
I20260812 06:56:04.940714 16877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:05.045727 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 16775
2026-08-12T06:56:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2727 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:05 dist-test-slave-6zbq krb5kdc[17753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:05 dist-test-slave-6zbq krb5kdc[17753](info): set up 2 sockets
Aug 12 06:56:05 dist-test-slave-6zbq krb5kdc[17753](info): commencing operation
krb5kdc: starting...
W20260812 06:56:06.279588 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.008s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:06 dist-test-slave-6zbq krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:06.539358 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38305
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:06.859421 17774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:06.859683 17774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:06.859772 17774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:06.866710 17774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:06.866798 17774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:06.866848 17774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:06.866904 17774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:06.866940 17774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:06.876140 17774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:06.877343 17774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:06.878623 17774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:06.889292 17783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:06.889331 17791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:06.889372 17789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:06.890380 17774 server_base.cc:1034] running on GCE node
I20260812 06:56:06.891095 17774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:06.892295 17774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:06.893488 17774 hybrid_clock.cc:648] HybridClock initialized: now 1786517766893437 us; error 38 us; skew 500 ppm
I20260812 06:56:06.893798 17774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:06 dist-test-slave-6zbq krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:06.896595 17774 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:06.897941 17774 webserver.cc:460] Webserver started at http://127.17.69.126:36539/ using document root <none> and password file <none>
I20260812 06:56:06.898597 17774 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:06.898710 17774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:06.898980 17774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:06.901753 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "f37ca758175b46b3b6cc7a7bedaeaa99"
format_stamp: "Formatted at 2026-08-12 06:56:06 on dist-test-slave-6zbq"
server_key: "51aa7ddfdae9fead72622c181a59a7fe"
server_key_iv: "2edcb2c40bdf47252eead7aabcbcdbdc"
server_key_version: "encryptionkey@0"
I20260812 06:56:06.902505 17774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "f37ca758175b46b3b6cc7a7bedaeaa99"
format_stamp: "Formatted at 2026-08-12 06:56:06 on dist-test-slave-6zbq"
server_key: "51aa7ddfdae9fead72622c181a59a7fe"
server_key_iv: "2edcb2c40bdf47252eead7aabcbcdbdc"
server_key_version: "encryptionkey@0"
I20260812 06:56:06.907078 17774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:56:06.910090 17801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:06.911316 17774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:06.911468 17774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "f37ca758175b46b3b6cc7a7bedaeaa99"
format_stamp: "Formatted at 2026-08-12 06:56:06 on dist-test-slave-6zbq"
server_key: "51aa7ddfdae9fead72622c181a59a7fe"
server_key_iv: "2edcb2c40bdf47252eead7aabcbcdbdc"
server_key_version: "encryptionkey@0"
I20260812 06:56:06.911631 17774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:06.939666 17774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:06.943300 17774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:06.943542 17774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:06.960176 17774 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38305
I20260812 06:56:06.960199 17890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38305 every 8 connection(s)
I20260812 06:56:06.961360 17774 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:06.962131 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17774
I20260812 06:56:06.962316 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:06.962769 17685 external_mini_cluster.cc:1428] Setting key 7b8057f5f0c3d4875848063230738dd4
I20260812 06:56:06.965273 17891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:56:06 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:06.977152 17891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: Bootstrap starting.
I20260812 06:56:06.980723 17891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:06.981933 17891 log.cc:824] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:06.984498 17891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: No bootstrap required, opened a new log
I20260812 06:56:06.985211 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:06.964401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.17.69.126:38305)
0812 06:56:06.965206 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:06.965221 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:06.965284 (+ 63us) server_negotiation.cc:407] Connection header received
0812 06:56:06.967513 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:06.967539 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:06.967987 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:06.968406 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:06.969694 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:06.971694 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:06.972944 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:06.973922 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:06.974098 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:06.977852 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:06.977863 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:06.977878 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:06.977924 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:06.981545 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:06.982161 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:06.982167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:06.982173 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:06.982326 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:06.982734 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:06.982739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:06.982741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:06.983230 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:06.983493 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:06.983941 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 06:56:06.984305 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":328,"threads_started":1}
I20260812 06:56:06.992015 17891 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } }
I20260812 06:56:06.992318 17891 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:06.992419 17891 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37ca758175b46b3b6cc7a7bedaeaa99, State: Initialized, Role: FOLLOWER
I20260812 06:56:06.992901 17891 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } }
I20260812 06:56:06.993106 17891 raft_consensus.cc:388] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:06.993233 17891 raft_consensus.cc:482] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:06.993384 17891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:06.995409 17891 raft_consensus.cc:504] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } }
I20260812 06:56:06.995843 17891 leader_election.cc:302] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f37ca758175b46b3b6cc7a7bedaeaa99; no voters:
I20260812 06:56:06.996376 17891 leader_election.cc:288] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:06.996459 17902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:06.997469 17902 raft_consensus.cc:686] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [term 1 LEADER]: Becoming Leader. State: Replica: f37ca758175b46b3b6cc7a7bedaeaa99, State: Running, Role: LEADER
I20260812 06:56:06.997915 17902 consensus_queue.cc:237] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } }
I20260812 06:56:06.998270 17891 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:07.000419 17903 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } } }
I20260812 06:56:07.000393 17906 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f37ca758175b46b3b6cc7a7bedaeaa99. Latest consensus state: current_term: 1 leader_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37ca758175b46b3b6cc7a7bedaeaa99" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38305 } } }
I20260812 06:56:07.000856 17903 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:07.000856 17906 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:07.002858 17916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:07.006488 17916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:07.014765 17916 catalog_manager.cc:1347] Generated new cluster ID: 319ac457aaf34ff8a33cf77a11350563
I20260812 06:56:07.014881 17916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:07.021296 17937 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 06:56:07.028934 17916 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:07.030076 17916 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:07.038290 17916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: Generated new TSK 0
I20260812 06:56:07.039206 17916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:56:07.107506 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:07.424317 17943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:07.424551 17943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:07.424681 17943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:07.431569 17943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:07.431661 17943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:07.431808 17943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:07.440913 17943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:07.441991 17943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:07.443215 17943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:07.455331 17954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.455371 17955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.455410 17961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:07.456326 17943 server_base.cc:1034] running on GCE node
I20260812 06:56:07.456866 17943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:07.457741 17943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:07.458940 17943 hybrid_clock.cc:648] HybridClock initialized: now 1786517767458893 us; error 33 us; skew 500 ppm
I20260812 06:56:07.459220 17943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:07 dist-test-slave-6zbq krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:07.462038 17943 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:56:07.463363 17943 webserver.cc:460] Webserver started at http://127.17.69.65:36191/ using document root <none> and password file <none>
I20260812 06:56:07.463909 17943 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:07.464002 17943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:07.464241 17943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:07.466809 17943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "bfa4f770b031434e91032abb2b145803"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "1d3b172769fb1733f471e5b55eab8d01"
server_key_iv: "41bcd5942a782b75edff7d6545382092"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.467471 17943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "bfa4f770b031434e91032abb2b145803"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "1d3b172769fb1733f471e5b55eab8d01"
server_key_iv: "41bcd5942a782b75edff7d6545382092"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.472002 17943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:07.475059 17974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.476194 17943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:07.476327 17943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "bfa4f770b031434e91032abb2b145803"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "1d3b172769fb1733f471e5b55eab8d01"
server_key_iv: "41bcd5942a782b75edff7d6545382092"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.476473 17943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:07.504257 17943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:07.507768 17943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:07.507987 17943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:07.508788 17943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:07.509974 17943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:07.510049 17943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.510136 17943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:07.510185 17943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.539348 17943 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:36063
I20260812 06:56:07.539394 18184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:36063 every 8 connection(s)
I20260812 06:56:07.540479 17943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:07.550724 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 17943
I20260812 06:56:07.550891 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:07.551293 17685 external_mini_cluster.cc:1428] Setting key 37113d0d43d13d19de5bcf9f7481a72b
Aug 12 06:56:07 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:07.559252 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:07.542714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:38981 (local address 127.17.69.126:38305)
0812 06:56:07.543154 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:07.543159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:07.543931 (+ 772us) server_negotiation.cc:407] Connection header received
0812 06:56:07.545684 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:07.545689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:07.545766 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:07.545864 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:07.547307 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.548012 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.549499 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.549888 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.550000 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:07.553425 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:07.553431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:07.553436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:07.553474 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:07.556388 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:07.557018 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:07.557022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:07.557026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:07.557170 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:07.557620 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:07.557625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:07.557628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:07.557948 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:07.558093 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:07.558737 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 06:56:07.558941 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":229,"threads_started":1}
I20260812 06:56:07.560000 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:07.543138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38305 (local address 127.17.69.65:38981)
0812 06:56:07.543771 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:07.543816 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:07.545420 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:07.546098 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:07.546106 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:07.546619 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:07.547133 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:07.547151 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.548163 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.548167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:07.549357 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:07.549371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:07.550103 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:07.550111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:07.550335 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:07.550976 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:07.551006 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:07.553220 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:07.556522 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:07.556528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:07.556538 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:07.556911 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:07.557285 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:07.557289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:07.557291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:07.557474 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:07.558060 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:07.558065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:07.558329 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 06:56:07.558627 (+ 298us) client_negotiation.cc:240] Negotiation successful
0812 06:56:07.558995 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":253,"threads_started":1}
I20260812 06:56:07.561342 18185 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38305
I20260812 06:56:07.561599 18185 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:07.562232 18185 heartbeater.cc:511] Master 127.17.69.126:38305 requested a full tablet report, sending...
I20260812 06:56:07.564281 17828 ts_manager.cc:194] Registered new tserver with Master: bfa4f770b031434e91032abb2b145803 (127.17.69.65:36063)
I20260812 06:56:07.565973 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:38981
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:56:07.610587 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:07.924401 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:07.924654 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:07.924769 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:07.931614 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:07.931717 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:07.931864 18200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:07.940864 18200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:07.941969 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:07.943185 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:07.955037 18209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.955415 18205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:07.955538 18212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:07.956194 18200 server_base.cc:1034] running on GCE node
I20260812 06:56:07.956779 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:07.957652 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:07.958856 18200 hybrid_clock.cc:648] HybridClock initialized: now 1786517767958814 us; error 35 us; skew 500 ppm
I20260812 06:56:07.959147 18200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:07 dist-test-slave-6zbq krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:07.967377 18200 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:56:07.968859 18200 webserver.cc:460] Webserver started at http://127.17.69.66:32771/ using document root <none> and password file <none>
I20260812 06:56:07.969470 18200 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:07.969581 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:07.969882 18200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:07.972718 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "bad09e60669945dcb4be95102ff313b2"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "d43047d323f46334f4d2369081b1ad57"
server_key_iv: "badb292d0e6b8b66980105db63f0c973"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.973417 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "bad09e60669945dcb4be95102ff313b2"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "d43047d323f46334f4d2369081b1ad57"
server_key_iv: "badb292d0e6b8b66980105db63f0c973"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.978178 18200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 06:56:07.981388 18224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:07.982561 18200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:07.982700 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "bad09e60669945dcb4be95102ff313b2"
format_stamp: "Formatted at 2026-08-12 06:56:07 on dist-test-slave-6zbq"
server_key: "d43047d323f46334f4d2369081b1ad57"
server_key_iv: "badb292d0e6b8b66980105db63f0c973"
server_key_version: "encryptionkey@0"
I20260812 06:56:07.982867 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:08.002889 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:08.006397 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:08.006639 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:08.007516 18200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:08.008728 18200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:08.008809 18200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.008924 18200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:08.008975 18200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.038412 18200 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:46443
I20260812 06:56:08.038468 18407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:46443 every 8 connection(s)
I20260812 06:56:08.039620 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:08.042680 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18200
I20260812 06:56:08.042915 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:08.043499 17685 external_mini_cluster.cc:1428] Setting key fe1a6df909de491edef81cbaab9b877d
Aug 12 06:56:08 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:08.059747 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.041901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:38531 (local address 127.17.69.126:38305)
0812 06:56:08.042117 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.042124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.043349 (+ 1225us) server_negotiation.cc:407] Connection header received
0812 06:56:08.045520 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.045525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.045599 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.045684 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.047142 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.047848 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.049373 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.049947 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.050073 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.053903 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.053910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.053915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.053950 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.056692 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.057467 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.057471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.057474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.057602 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.058018 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.058022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.058025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.058301 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.058445 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.059179 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.059419 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 06:56:08.060575 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.042372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38305 (local address 127.17.69.66:38531)
0812 06:56:08.043176 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.043227 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.045273 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.045914 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.045922 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.046456 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:08.046978 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.046996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.047994 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.047998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.049207 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.049221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.050134 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.050139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.050417 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.051132 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:08.051158 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:08.053687 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:08.056826 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.056834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.056848 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.057367 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.057694 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.057698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.057701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.057928 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.058426 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:08.058433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:08.058812 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.059036 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.059513 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":291,"threads_started":1}
I20260812 06:56:08.061887 18408 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38305
I20260812 06:56:08.062153 18408 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:08.062868 18408 heartbeater.cc:511] Master 127.17.69.126:38305 requested a full tablet report, sending...
I20260812 06:56:08.064280 17828 ts_manager.cc:194] Registered new tserver with Master: bad09e60669945dcb4be95102ff313b2 (127.17.69.66:46443)
I20260812 06:56:08.065035 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:38531
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:56:08.106105 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:37561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:08.417070 18418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:08.417328 18418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:08.417446 18418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:08.424410 18418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:08.424517 18418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:08.424665 18418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:08.433626 18418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:37561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:08.434732 18418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:08.435868 18418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:08.448954 18425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:08.449463 18433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:08.449874 18435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:08.450464 18418 server_base.cc:1034] running on GCE node
I20260812 06:56:08.450973 18418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:08.451767 18418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:08.452963 18418 hybrid_clock.cc:648] HybridClock initialized: now 1786517768452918 us; error 38 us; skew 500 ppm
I20260812 06:56:08.453253 18418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:08 dist-test-slave-6zbq krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:08.456214 18418 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:56:08.458269 18418 webserver.cc:460] Webserver started at http://127.17.69.67:36003/ using document root <none> and password file <none>
I20260812 06:56:08.458915 18418 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:08.459038 18418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:08.459324 18418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:08.462101 18418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "5721ac5c014d47b4b6e6d27cdee27dc7"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-6zbq"
server_key: "c0b40d1d911dda6ed0f34cb8ae2410db"
server_key_iv: "0d8d38b6642a91f35c49b680ab508cea"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.462841 18418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "5721ac5c014d47b4b6e6d27cdee27dc7"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-6zbq"
server_key: "c0b40d1d911dda6ed0f34cb8ae2410db"
server_key_iv: "0d8d38b6642a91f35c49b680ab508cea"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.467453 18418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:08.470582 18445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.471735 18418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:08.471875 18418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "5721ac5c014d47b4b6e6d27cdee27dc7"
format_stamp: "Formatted at 2026-08-12 06:56:08 on dist-test-slave-6zbq"
server_key: "c0b40d1d911dda6ed0f34cb8ae2410db"
server_key_iv: "0d8d38b6642a91f35c49b680ab508cea"
server_key_version: "encryptionkey@0"
I20260812 06:56:08.472043 18418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:08.482975 18418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:08.486915 18418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:08.487145 18418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:08.487965 18418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:08.489144 18418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:08.489219 18418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.489337 18418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:08.489389 18418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:08.517385 18418 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:37051
I20260812 06:56:08.517576 18646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:37051 every 8 connection(s)
I20260812 06:56:08.518707 18418 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:08.520093 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18418
I20260812 06:56:08.520267 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:08.520704 17685 external_mini_cluster.cc:1428] Setting key ea9e2737bb37f044fad96692840e3af1
Aug 12 06:56:08 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:08.536551 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.520989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:55041 (local address 127.17.69.126:38305)
0812 06:56:08.521203 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.521208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.522498 (+ 1290us) server_negotiation.cc:407] Connection header received
0812 06:56:08.524400 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.524404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.524467 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.524546 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.525823 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.526602 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.527987 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.528390 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.528475 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.531488 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.531493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.531497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.531524 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.533690 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.534448 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.534453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.534457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.534594 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.535055 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.535059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.535062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.535424 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.535562 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.536153 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.536304 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 06:56:08.537271 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.521542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38305 (local address 127.17.69.67:55041)
0812 06:56:08.522316 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.522386 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.524155 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.524718 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.524726 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.525198 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:08.525703 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.525720 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.526721 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.526725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.527881 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.527892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.528542 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.528547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.528719 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.529292 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:08.529313 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:08.531332 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:08.533813 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.533821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.533835 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.534322 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.534699 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:08.534704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:08.534707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:08.534930 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:08.535567 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:08.535574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:08.535859 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.536059 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.536371 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":298,"threads_started":1}
I20260812 06:56:08.538326 18647 heartbeater.cc:347] Connected to a master server at 127.17.69.126:38305
I20260812 06:56:08.538636 18647 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:08.539227 18647 heartbeater.cc:511] Master 127.17.69.126:38305 requested a full tablet report, sending...
I20260812 06:56:08.540469 17828 ts_manager.cc:194] Registered new tserver with Master: 5721ac5c014d47b4b6e6d27cdee27dc7 (127.17.69.67:37051)
I20260812 06:56:08.541218 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:55041
I20260812 06:56:08.547358 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:08.563107 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.551115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.17.69.126:38305)
0812 06:56:08.551257 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.551262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.551646 (+ 384us) server_negotiation.cc:407] Connection header received
0812 06:56:08.551788 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.551793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.551848 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.551922 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.553297 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.553967 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.555277 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.555668 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.555774 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.558226 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.558233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.558238 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.558269 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.560508 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.561231 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.561236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.561239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.561373 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.561782 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.561786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.561788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.562065 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.562303 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.562692 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.562852 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 06:56:08.568569 18185 heartbeater.cc:503] Master 127.17.69.126:38305 was elected leader, sending a full tablet report...
I20260812 06:56:08.566300 17828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:08.572929 17828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:08.594421 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.584010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.17.69.65:36063)
0812 06:56:08.586696 (+ 2686us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.586705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.586735 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:56:08.586843 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.586852 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.587110 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.587299 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:08.587950 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.589523 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.592042 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.593210 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.593332 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.593551 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.593914 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.594075 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":2344,"threads_started":1}
I20260812 06:56:08.594422 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.583680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:36063 (local address 127.0.0.1:51840)
0812 06:56:08.584591 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.584632 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.584818 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.587374 (+ 2556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.587382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.587414 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:08.587760 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.587772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.589692 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.589697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.591865 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.591880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.593539 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.593552 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.593714 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.593747 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.593865 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.594048 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":545,"threads_started":1}
I20260812 06:56:08.597587 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.584296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46443 (local address 127.0.0.1:50248)
0812 06:56:08.585984 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.586002 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.586117 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.590481 (+ 4364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.590486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.590515 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:08.591110 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.591121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.592764 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.592769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.595065 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.595084 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.596793 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.596800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.596959 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.596997 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.597082 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.597201 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1557,"mutex_wait_us":174,"thread_start_us":1417,"threads_started":1}
I20260812 06:56:08.597771 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.584873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.17.69.66:46443)
0812 06:56:08.589911 (+ 5038us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.589920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.589945 (+ 25us) server_negotiation.cc:407] Connection header received
0812 06:56:08.590035 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.590042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.590247 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.590650 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:08.591289 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.592633 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.595218 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.596581 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.596719 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.596892 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.597312 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.597466 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4865,"thread_start_us":4735,"threads_started":1}
I20260812 06:56:08.598147 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.585450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.17.69.67:37051)
0812 06:56:08.589198 (+ 3748us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.589207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.589881 (+ 674us) server_negotiation.cc:407] Connection header received
0812 06:56:08.589952 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.589957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.590103 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.590249 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:08.591201 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.592688 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.595266 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.596874 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.597003 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.597203 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.597687 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.597839 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3569,"thread_start_us":3411,"threads_started":1}
I20260812 06:56:08.598444 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.585002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:37051 (local address 127.0.0.1:51360)
0812 06:56:08.589623 (+ 4621us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.589641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.589763 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.590274 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.590279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.590367 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:08.591006 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.591023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.592788 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.592793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.595095 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.595105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.597059 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.597066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.597220 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.597250 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.597344 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.597458 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4505,"mutex_wait_us":53,"thread_start_us":4351,"threads_started":1}
I20260812 06:56:08.598470 18059 tablet_service.cc:1467] Processing CreateTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 (DEFAULT_TABLE table=test-table [id=492c93acc5a04ca9ac16874beb2c8e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:08.599634 18059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b81aeeadd34a4e89b3114e3cab7a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:08.600715 18299 tablet_service.cc:1467] Processing CreateTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 (DEFAULT_TABLE table=test-table [id=492c93acc5a04ca9ac16874beb2c8e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:08.601814 18299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b81aeeadd34a4e89b3114e3cab7a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:08.605406 18527 tablet_service.cc:1467] Processing CreateTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 (DEFAULT_TABLE table=test-table [id=492c93acc5a04ca9ac16874beb2c8e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:08.606706 18527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b81aeeadd34a4e89b3114e3cab7a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:08.616029 18674 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Bootstrap starting.
I20260812 06:56:08.617821 18675 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Bootstrap starting.
I20260812 06:56:08.619122 18674 tablet_bootstrap.cc:654] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:08.620008 18677 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Bootstrap starting.
I20260812 06:56:08.620348 18674 log.cc:824] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:08.621243 18675 tablet_bootstrap.cc:654] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:08.622298 18675 log.cc:824] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:08.622977 18677 tablet_bootstrap.cc:654] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:08.623505 18674 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: No bootstrap required, opened a new log
I20260812 06:56:08.623775 18674 ts_tablet_manager.cc:1397] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 06:56:08.624125 18677 log.cc:824] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:08.625232 18675 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: No bootstrap required, opened a new log
I20260812 06:56:08.625480 18675 ts_tablet_manager.cc:1397] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 06:56:08.626981 18677 tablet_bootstrap.cc:492] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: No bootstrap required, opened a new log
I20260812 06:56:08.627214 18677 ts_tablet_manager.cc:1397] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 06:56:08.630883 18674 raft_consensus.cc:348] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.631194 18674 raft_consensus.cc:374] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:08.631297 18674 raft_consensus.cc:729] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad09e60669945dcb4be95102ff313b2, State: Initialized, Role: FOLLOWER
I20260812 06:56:08.631819 18674 consensus_queue.cc:260] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.632762 18408 heartbeater.cc:503] Master 127.17.69.126:38305 was elected leader, sending a full tablet report...
I20260812 06:56:08.634456 18674 ts_tablet_manager.cc:1428] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260812 06:56:08.635262 18675 raft_consensus.cc:348] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.635653 18675 raft_consensus.cc:374] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:08.635746 18675 raft_consensus.cc:729] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa4f770b031434e91032abb2b145803, State: Initialized, Role: FOLLOWER
I20260812 06:56:08.636291 18675 consensus_queue.cc:260] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.636962 18677 raft_consensus.cc:348] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.637355 18677 raft_consensus.cc:374] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:08.637454 18677 raft_consensus.cc:729] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5721ac5c014d47b4b6e6d27cdee27dc7, State: Initialized, Role: FOLLOWER
I20260812 06:56:08.637938 18675 ts_tablet_manager.cc:1428] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260812 06:56:08.638007 18677 consensus_queue.cc:260] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.642957 18647 heartbeater.cc:503] Master 127.17.69.126:38305 was elected leader, sending a full tablet report...
I20260812 06:56:08.643383 18677 ts_tablet_manager.cc:1428] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 06:56:08.719954 18692 raft_consensus.cc:482] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:08.720175 18692 raft_consensus.cc:504] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.721699 18692 leader_election.cc:288] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa4f770b031434e91032abb2b145803 (127.17.69.65:36063), bad09e60669945dcb4be95102ff313b2 (127.17.69.66:46443)
I20260812 06:56:08.727650 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.722246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:48969 (local address 127.17.69.66:46443)
0812 06:56:08.722470 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.722476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.722495 (+ 19us) server_negotiation.cc:407] Connection header received
0812 06:56:08.722561 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.722566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.722624 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.722699 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:08.723120 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.724032 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.725818 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.726901 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.727015 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.727065 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.727388 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.727470 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:56:08.727774 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.721942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:46443 (local address 127.17.69.67:48969)
0812 06:56:08.722280 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.722299 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.722437 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.722745 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.722750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.722786 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:08.722976 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.722989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.724157 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.724163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.725667 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.725678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.727041 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.727047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.727232 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.727277 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.727373 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.727504 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":20}
I20260812 06:56:08.728603 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.722095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45437 (local address 127.17.69.65:36063)
0812 06:56:08.722279 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.722286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.722918 (+ 632us) server_negotiation.cc:407] Connection header received
0812 06:56:08.723049 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.723055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.723128 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.723209 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:08.723764 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.724691 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.726655 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.727671 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.727824 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.727888 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.728263 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.728381 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:56:08.728688 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.722061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:36063 (local address 127.17.69.67:45437)
0812 06:56:08.722814 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:08.722835 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:08.722938 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:08.723248 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:08.723252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:08.723278 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:08.723622 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.723634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.724848 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.724852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.726511 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:08.726526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.727974 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.727980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:08.728108 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.728135 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 06:56:08.728214 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 06:56:08.728336 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":1,"thread_start_us":453,"threads_started":1}
I20260812 06:56:08.728663 18331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b81aeeadd34a4e89b3114e3cab7a36" candidate_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad09e60669945dcb4be95102ff313b2" is_pre_election: true
I20260812 06:56:08.729148 18331 raft_consensus.cc:2454] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5721ac5c014d47b4b6e6d27cdee27dc7 in term 0.
I20260812 06:56:08.729508 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b81aeeadd34a4e89b3114e3cab7a36" candidate_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa4f770b031434e91032abb2b145803" is_pre_election: true
I20260812 06:56:08.729743 18453 leader_election.cc:302] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5721ac5c014d47b4b6e6d27cdee27dc7, bad09e60669945dcb4be95102ff313b2; no voters:
I20260812 06:56:08.729987 18095 raft_consensus.cc:2454] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5721ac5c014d47b4b6e6d27cdee27dc7 in term 0.
I20260812 06:56:08.730087 18692 raft_consensus.cc:2781] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:08.730222 18692 raft_consensus.cc:482] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:08.730332 18692 raft_consensus.cc:3037] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:08.732455 18692 raft_consensus.cc:504] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.732993 18692 leader_election.cc:288] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [CANDIDATE]: Term 1 election: Requested vote from peers bfa4f770b031434e91032abb2b145803 (127.17.69.65:36063), bad09e60669945dcb4be95102ff313b2 (127.17.69.66:46443)
I20260812 06:56:08.733284 18095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b81aeeadd34a4e89b3114e3cab7a36" candidate_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa4f770b031434e91032abb2b145803"
I20260812 06:56:08.733443 18095 raft_consensus.cc:3037] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:08.733582 18331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b81aeeadd34a4e89b3114e3cab7a36" candidate_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad09e60669945dcb4be95102ff313b2"
I20260812 06:56:08.733795 18331 raft_consensus.cc:3037] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:08.736506 18095 raft_consensus.cc:2454] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5721ac5c014d47b4b6e6d27cdee27dc7 in term 1.
I20260812 06:56:08.736819 18331 raft_consensus.cc:2454] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5721ac5c014d47b4b6e6d27cdee27dc7 in term 1.
I20260812 06:56:08.736884 18449 leader_election.cc:302] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5721ac5c014d47b4b6e6d27cdee27dc7, bfa4f770b031434e91032abb2b145803; no voters:
I20260812 06:56:08.737169 18692 raft_consensus.cc:2781] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:08.737550 18692 raft_consensus.cc:686] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 1 LEADER]: Becoming Leader. State: Replica: 5721ac5c014d47b4b6e6d27cdee27dc7, State: Running, Role: LEADER
I20260812 06:56:08.737952 18692 consensus_queue.cc:237] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } }
I20260812 06:56:08.741391 17828 catalog_manager.cc:5495] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5721ac5c014d47b4b6e6d27cdee27dc7 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5721ac5c014d47b4b6e6d27cdee27dc7" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 37051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 } health_report { overall_health: UNKNOWN } } }
W20260812 06:56:08.770152 18648 tablet.cc:2369] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:08.782554 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.777892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.17.69.67:37051)
0812 06:56:08.778061 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.778065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.778081 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:56:08.778248 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.778252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.778323 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.778447 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:08.778767 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.779431 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.780727 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.781113 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.781200 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.781610 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:08.782028 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:08.782150 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.782251 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.782296 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:56:08.789589 18095 raft_consensus.cc:1261] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 1 FOLLOWER]: Refusing update from remote peer 5721ac5c014d47b4b6e6d27cdee27dc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:08.789590 18331 raft_consensus.cc:1261] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 1 FOLLOWER]: Refusing update from remote peer 5721ac5c014d47b4b6e6d27cdee27dc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:08.790421 18697 consensus_queue.cc:1035] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa4f770b031434e91032abb2b145803" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 36063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:08.790736 18692 consensus_queue.cc:1035] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad09e60669945dcb4be95102ff313b2" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:56:08.791404 18409 tablet.cc:2369] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:08.792651 18186 tablet.cc:2369] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:08.801339 18702 mvcc.cc:205] Tried to move back new op lower bound from 7317576780953874432 to 7317576780754243584. Current Snapshot: MvccSnapshot[applied={T|T < 7317576780953874432}]
I20260812 06:56:08.814168 17826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:08.814445 17826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:08.817875 17826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37ca758175b46b3b6cc7a7bedaeaa99: Sending DeleteTablet for 3 replicas of tablet 38b81aeeadd34a4e89b3114e3cab7a36
I20260812 06:56:08.818918 18059 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:08 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:51840
I20260812 06:56:08.818961 18527 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:08 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:51360
I20260812 06:56:08.819337 18299 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b81aeeadd34a4e89b3114e3cab7a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:08 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:50248
I20260812 06:56:08.822445 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17943
I20260812 06:56:08.841277 18717 tablet_replica.cc:330] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: stopping tablet replica
I20260812 06:56:08.841878 18719 tablet_replica.cc:330] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: stopping tablet replica
I20260812 06:56:08.842319 18719 raft_consensus.cc:2229] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:08.842885 18717 raft_consensus.cc:2229] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:08.843439 18717 raft_consensus.cc:2258] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:08.844398 18722 tablet_replica.cc:330] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: stopping tablet replica
I20260812 06:56:08.844812 18722 raft_consensus.cc:2229] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:08.845017 18719 raft_consensus.cc:2258] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:08.845373 18722 raft_consensus.cc:2258] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:08.846127 18717 ts_tablet_manager.cc:1905] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:08.847458 18722 ts_tablet_manager.cc:1905] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:08.847851 18719 ts_tablet_manager.cc:1905] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:08.852873 18717 ts_tablet_manager.cc:1918] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:08.853029 18717 log.cc:1196] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/38b81aeeadd34a4e89b3114e3cab7a36
I20260812 06:56:08.853571 18717 ts_tablet_manager.cc:1939] T 38b81aeeadd34a4e89b3114e3cab7a36 P bfa4f770b031434e91032abb2b145803: Deleting consensus metadata
Aug 12 06:56:08 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:56:08.854419 18722 ts_tablet_manager.cc:1918] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:08.854650 18722 log.cc:1196] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/38b81aeeadd34a4e89b3114e3cab7a36
I20260812 06:56:08.855204 18722 ts_tablet_manager.cc:1939] T 38b81aeeadd34a4e89b3114e3cab7a36 P bad09e60669945dcb4be95102ff313b2: Deleting consensus metadata
I20260812 06:56:08.855365 17803 catalog_manager.cc:4841] TS bfa4f770b031434e91032abb2b145803 (127.17.69.65:36063): tablet 38b81aeeadd34a4e89b3114e3cab7a36 (table test-table [id=492c93acc5a04ca9ac16874beb2c8e30]) successfully deleted
I20260812 06:56:08.856868 17805 catalog_manager.cc:4841] TS bad09e60669945dcb4be95102ff313b2 (127.17.69.66:46443): tablet 38b81aeeadd34a4e89b3114e3cab7a36 (table test-table [id=492c93acc5a04ca9ac16874beb2c8e30]) successfully deleted
I20260812 06:56:08.858776 18719 ts_tablet_manager.cc:1918] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:08.858946 18719 log.cc:1196] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/38b81aeeadd34a4e89b3114e3cab7a36
I20260812 06:56:08.859480 18719 ts_tablet_manager.cc:1939] T 38b81aeeadd34a4e89b3114e3cab7a36 P 5721ac5c014d47b4b6e6d27cdee27dc7: Deleting consensus metadata
I20260812 06:56:08.861003 17803 catalog_manager.cc:4841] TS 5721ac5c014d47b4b6e6d27cdee27dc7 (127.17.69.67:37051): tablet 38b81aeeadd34a4e89b3114e3cab7a36 (table test-table [id=492c93acc5a04ca9ac16874beb2c8e30]) successfully deleted
I20260812 06:56:08.861426 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.823746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.17.69.65:36063)
0812 06:56:08.823926 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.823933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.835178 (+ 11245us) server_negotiation.cc:407] Connection header received
0812 06:56:08.835587 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.835595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.835677 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.835745 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.837516 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.838851 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.842648 (+ 3797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.843393 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.843524 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.855585 (+ 12061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.855599 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.855615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.855653 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.858690 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.859432 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.859439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.859441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.859580 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.859954 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.859959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.859963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.860301 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.860571 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.860906 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.861137 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:56:08.862504 18173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:08.948573 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17943
I20260812 06:56:08.971843 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18200
Aug 12 06:56:08 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:56:08.986564 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:08.972613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.17.69.66:46443)
0812 06:56:08.972804 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:08.972810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:08.972900 (+ 90us) server_negotiation.cc:407] Connection header received
0812 06:56:08.973075 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:08.973079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:08.973154 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:08.973237 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:08.974536 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.975296 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.976655 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:08.977046 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:08.977154 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:08.981132 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:08.981146 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:08.981162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:08.981194 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:08.983851 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.984501 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.984508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.984511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.984632 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:08.984985 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:08.984989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:08.984991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:08.985343 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:08.985623 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:08.985966 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 06:56:08.986182 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 06:56:08.988044 18399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:09.144505 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18200
I20260812 06:56:09.167636 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18418
Aug 12 06:56:09 dist-test-slave-6zbq krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:56:09.185762 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.168388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.17.69.67:37051)
0812 06:56:09.168572 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.168576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.168615 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:09.168819 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.168823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.168885 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.168965 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.170722 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.171515 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.173262 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.173699 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.173787 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.179052 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.179067 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.179088 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:09.179132 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.182994 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.183618 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.183628 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.183631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.183858 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:09.184181 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:09.184187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:09.184190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:09.184631 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.184980 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.185141 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.185359 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:56:09.186549 18637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:09.305102 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18418
I20260812 06:56:09.329885 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 17774
I20260812 06:56:09.330938 17884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:09.438486 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 17774
2026-08-12T06:56:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4394 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T06:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:09Z Disabled control of system clock
I20260812 06:56:09.524533 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34795
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:34795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:09.847112 18736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:09.847364 18736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:09.847447 18736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:09.854250 18736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:09.854357 18736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:09.854419 18736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:09.854491 18736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:09.854549 18736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:09.863229 18736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:34795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:34795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:09.864305 18736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:09.865933 18736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:09.876840 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:09.876868 18744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:09.877336 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:09.877604 18736 server_base.cc:1034] running on GCE node
I20260812 06:56:09.878273 18736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:09.879499 18736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:09.880695 18736 hybrid_clock.cc:648] HybridClock initialized: now 1786517769880649 us; error 36 us; skew 500 ppm
I20260812 06:56:09.881023 18736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:09.883128 18736 webserver.cc:460] Webserver started at http://127.17.69.126:40595/ using document root <none> and password file <none>
I20260812 06:56:09.883704 18736 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:09.883803 18736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:09.884091 18736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:09.886791 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "8be90f6166c64cec8c8ea701a62b02bb"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-6zbq"
server_key: "a57758986ddb07068a9d30787ddf0cb2"
server_key_iv: "4bba6d3fb5e60d3ce7ee8c6306b5b8d9"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.887521 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "8be90f6166c64cec8c8ea701a62b02bb"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-6zbq"
server_key: "a57758986ddb07068a9d30787ddf0cb2"
server_key_iv: "4bba6d3fb5e60d3ce7ee8c6306b5b8d9"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.892043 18736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 06:56:09.895052 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:09.896149 18736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:09.896297 18736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "8be90f6166c64cec8c8ea701a62b02bb"
format_stamp: "Formatted at 2026-08-12 06:56:09 on dist-test-slave-6zbq"
server_key: "a57758986ddb07068a9d30787ddf0cb2"
server_key_iv: "4bba6d3fb5e60d3ce7ee8c6306b5b8d9"
server_key_version: "encryptionkey@0"
I20260812 06:56:09.896482 18736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:09.915273 18736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:09.915966 18736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:09.916198 18736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:09.932595 18736 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:34795
I20260812 06:56:09.932626 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:34795 every 8 connection(s)
I20260812 06:56:09.933703 18736 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:09.936955 18852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:09.938259 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18736
I20260812 06:56:09.938460 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:09.939023 17685 external_mini_cluster.cc:1428] Setting key 8f5d72b247f12d2ca0b71a5257f52698
I20260812 06:56:09.949127 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: Bootstrap starting.
I20260812 06:56:09.951334 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:09.940955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.17.69.126:34795)
0812 06:56:09.941916 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:09.941929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:09.941970 (+ 41us) server_negotiation.cc:407] Connection header received
0812 06:56:09.943526 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:09.943547 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:09.943966 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:09.944292 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:09.944848 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.946033 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.947392 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:09.948005 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:09.948163 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:09.949321 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:09.949328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:09.949344 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:09.949810 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:09.949884 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:09.949904 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:09.950011 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:09.950218 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 06:56:09.950511 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":332,"threads_started":1}
I20260812 06:56:09.952152 18852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:09.953392 18852 log.cc:824] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:09.955940 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: No bootstrap required, opened a new log
I20260812 06:56:09.963116 18852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } }
I20260812 06:56:09.963517 18852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:09.963594 18852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be90f6166c64cec8c8ea701a62b02bb, State: Initialized, Role: FOLLOWER
I20260812 06:56:09.964036 18852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } }
I20260812 06:56:09.964231 18852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:09.964337 18852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:09.964447 18852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:09.966322 18852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } }
I20260812 06:56:09.966770 18852 leader_election.cc:302] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8be90f6166c64cec8c8ea701a62b02bb; no voters:
I20260812 06:56:09.967258 18852 leader_election.cc:288] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:09.967399 18858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:09.968094 18858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [term 1 LEADER]: Becoming Leader. State: Replica: 8be90f6166c64cec8c8ea701a62b02bb, State: Running, Role: LEADER
I20260812 06:56:09.968590 18858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } }
I20260812 06:56:09.969197 18852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:09.971366 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8be90f6166c64cec8c8ea701a62b02bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } } }
I20260812 06:56:09.971330 18861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8be90f6166c64cec8c8ea701a62b02bb. Latest consensus state: current_term: 1 leader_uuid: "8be90f6166c64cec8c8ea701a62b02bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be90f6166c64cec8c8ea701a62b02bb" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 34795 } } }
I20260812 06:56:09.971808 18861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:09.971786 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:09.978953 18873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:09.983597 18873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:09.991024 18873 catalog_manager.cc:1347] Generated new cluster ID: 02931a2f90fb4b08be5845836b756c60
I20260812 06:56:09.991127 18873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:10.007148 18873 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:10.008826 18873 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:10.015810 18873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: Generated new TSK 0
I20260812 06:56:10.016654 18873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:10.023555 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--builtin_ntp_servers=127.17.69.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:10.346702 18890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:10.346935 18890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:10.347016 18890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:10.353870 18890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:10.353953 18890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:10.354053 18890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:10.362650 18890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:10.363732 18890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:10.365339 18890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:10.377619 18903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:10.377669 18900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:10.377704 18897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:10.378466 18890 server_base.cc:1034] running on GCE node
I20260812 06:56:10.379005 18890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:10.379877 18890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:10.381088 18890 hybrid_clock.cc:648] HybridClock initialized: now 1786517770381018 us; error 56 us; skew 500 ppm
I20260812 06:56:10.381386 18890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:10.383423 18890 webserver.cc:460] Webserver started at http://127.17.69.65:35631/ using document root <none> and password file <none>
I20260812 06:56:10.384016 18890 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:10.384106 18890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:10.384395 18890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:10.387085 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "00f0da14cf3aa08dcc7b83eeeca9faad"
server_key_iv: "5e9f29dd04be7b4d2fd324dd404bd91a"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.387791 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "00f0da14cf3aa08dcc7b83eeeca9faad"
server_key_iv: "5e9f29dd04be7b4d2fd324dd404bd91a"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.392321 18890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:10.395397 18913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.396548 18890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:10.396688 18890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "00f0da14cf3aa08dcc7b83eeeca9faad"
server_key_iv: "5e9f29dd04be7b4d2fd324dd404bd91a"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.396868 18890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:10.407964 18890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:10.408629 18890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:10.408885 18890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:10.409734 18890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:10.411058 18890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:10.411162 18890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.411271 18890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:10.411339 18890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.440922 18890 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:40899
I20260812 06:56:10.440973 19119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:40899 every 8 connection(s)
I20260812 06:56:10.442157 18890 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:10.443202 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 18890
I20260812 06:56:10.443408 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:10.443991 17685 external_mini_cluster.cc:1428] Setting key 2adaf03ee5108aa7e651a9c4c683d087
I20260812 06:56:10.446823 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--builtin_ntp_servers=127.17.69.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:10.454802 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:10.444813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:38223 (local address 127.17.69.126:34795)
0812 06:56:10.444986 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:10.444992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:10.446336 (+ 1344us) server_negotiation.cc:407] Connection header received
0812 06:56:10.448111 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:10.448117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:10.448197 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:10.448282 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:10.449270 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.449985 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.451887 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.452416 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.452501 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:10.453976 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:10.453982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:10.453985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:10.454002 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:10.454043 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:10.454055 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:10.454130 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:10.454365 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 06:56:10.454468 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:56:10.455675 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:10.445263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34795 (local address 127.17.69.65:38223)
0812 06:56:10.446199 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:10.446235 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:10.447860 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:10.448501 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:10.448509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:10.448593 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:10.449107 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:10.449123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.450126 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.450131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:10.451727 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:10.451741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.452800 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.452807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:10.453053 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:10.453676 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:10.453679 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:10.453722 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:10.453793 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:10.454143 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:10.454155 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:10.454162 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:56:10.454320 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 06:56:10.454698 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":299,"threads_started":1}
I20260812 06:56:10.457021 19120 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34795
I20260812 06:56:10.457298 19120 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:10.457949 19120 heartbeater.cc:511] Master 127.17.69.126:34795 requested a full tablet report, sending...
I20260812 06:56:10.459918 18791 ts_manager.cc:194] Registered new tserver with Master: 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 (127.17.69.65:40899)
I20260812 06:56:10.461369 18791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.65:38223
W20260812 06:56:10.776083 19124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:10.776317 19124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:10.776435 19124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:10.783427 19124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:10.783511 19124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:10.783648 19124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:10.792187 19124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:10.793289 19124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:10.794940 19124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:10.807016 19131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:10.807089 19130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:10.807053 19133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:10.807773 19124 server_base.cc:1034] running on GCE node
I20260812 06:56:10.808303 19124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:10.809118 19124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:10.810316 19124 hybrid_clock.cc:648] HybridClock initialized: now 1786517770810275 us; error 31 us; skew 500 ppm
I20260812 06:56:10.810634 19124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:10.812623 19124 webserver.cc:460] Webserver started at http://127.17.69.66:43885/ using document root <none> and password file <none>
I20260812 06:56:10.813238 19124 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:10.813330 19124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:10.813616 19124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:10.816280 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "65837c1f6ebe4442ab16a592eb27fb30"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "15a2221fa3af612d89d26c0d38c76bfa"
server_key_iv: "4837852a2e8469690b8545ae0247f8ec"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.816994 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "65837c1f6ebe4442ab16a592eb27fb30"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "15a2221fa3af612d89d26c0d38c76bfa"
server_key_iv: "4837852a2e8469690b8545ae0247f8ec"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.821544 19124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:10.824663 19147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.825822 19124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:56:10.825958 19124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "65837c1f6ebe4442ab16a592eb27fb30"
format_stamp: "Formatted at 2026-08-12 06:56:10 on dist-test-slave-6zbq"
server_key: "15a2221fa3af612d89d26c0d38c76bfa"
server_key_iv: "4837852a2e8469690b8545ae0247f8ec"
server_key_version: "encryptionkey@0"
I20260812 06:56:10.826144 19124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:10.849090 19124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:10.849793 19124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:10.850059 19124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:10.850929 19124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:10.852253 19124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:10.852341 19124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.852442 19124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:10.852509 19124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:10.881374 19124 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:34009
I20260812 06:56:10.881488 19330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:34009 every 8 connection(s)
I20260812 06:56:10.882587 19124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:10.886610 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19124
I20260812 06:56:10.886799 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:10.887363 17685 external_mini_cluster.cc:1428] Setting key 3f88083589854b07a3f8462712ed41d0
I20260812 06:56:10.890293 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--builtin_ntp_servers=127.17.69.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:10.895457 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:10.884930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:41125 (local address 127.17.69.126:34795)
0812 06:56:10.885124 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:10.885129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:10.886464 (+ 1335us) server_negotiation.cc:407] Connection header received
0812 06:56:10.888648 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:10.888653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:10.888717 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:10.888796 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:10.889993 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.890806 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.892863 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.893276 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.893368 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:10.894777 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:10.894783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:10.894787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:10.894808 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:10.894849 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:10.894866 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:10.894950 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:10.895165 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 06:56:10.895238 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:56:10.896621 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:10.885412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34795 (local address 127.17.69.66:41125)
0812 06:56:10.886270 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:10.886323 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:10.888376 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:10.889034 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:10.889042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:10.889133 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:10.889828 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:10.889852 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.890965 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.890970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:10.892720 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:10.892735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:10.893482 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:10.893488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:10.893753 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:10.894438 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:10.894441 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:10.894486 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:10.894579 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:10.894972 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:10.894983 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:10.894990 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:56:10.895145 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 06:56:10.895492 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":264,"threads_started":1}
I20260812 06:56:10.897876 19332 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34795
I20260812 06:56:10.898168 19332 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:10.898998 19332 heartbeater.cc:511] Master 127.17.69.126:34795 requested a full tablet report, sending...
I20260812 06:56:10.900408 18791 ts_manager.cc:194] Registered new tserver with Master: 65837c1f6ebe4442ab16a592eb27fb30 (127.17.69.66:34009)
I20260812 06:56:10.901381 18791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.66:41125
W20260812 06:56:11.213814 19339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:11.214067 19339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:11.214186 19339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:11.220980 19339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:11.221071 19339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:11.221235 19339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:11.229889 19339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:34795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:11.231029 19339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:11.232642 19339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:11.244302 19350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:11.244292 19352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:11.244525 19354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:11.245087 19339 server_base.cc:1034] running on GCE node
I20260812 06:56:11.245595 19339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:11.246438 19339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:11.247637 19339 hybrid_clock.cc:648] HybridClock initialized: now 1786517771247586 us; error 41 us; skew 500 ppm
I20260812 06:56:11.247907 19339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:11.249938 19339 webserver.cc:460] Webserver started at http://127.17.69.67:40367/ using document root <none> and password file <none>
I20260812 06:56:11.250568 19339 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:11.250674 19339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:11.250952 19339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:11.253484 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "f8cd079999b54ba497c9e55003284862"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-6zbq"
server_key: "fccf25261c5a96d0d2f019fcf91388e3"
server_key_iv: "0b4e53817271b1a8852ab4e2d4278541"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.254202 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "f8cd079999b54ba497c9e55003284862"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-6zbq"
server_key: "fccf25261c5a96d0d2f019fcf91388e3"
server_key_iv: "0b4e53817271b1a8852ab4e2d4278541"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.258752 19339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:11.261749 19363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:11.262910 19339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:11.263051 19339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "f8cd079999b54ba497c9e55003284862"
format_stamp: "Formatted at 2026-08-12 06:56:11 on dist-test-slave-6zbq"
server_key: "fccf25261c5a96d0d2f019fcf91388e3"
server_key_iv: "0b4e53817271b1a8852ab4e2d4278541"
server_key_version: "encryptionkey@0"
I20260812 06:56:11.263232 19339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:11.306995 19339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:11.307825 19339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:11.308102 19339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:11.309077 19339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:11.310526 19339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:11.310631 19339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:11.310736 19339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:11.310813 19339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:11.339934 19339 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:38013
I20260812 06:56:11.340008 19559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:38013 every 8 connection(s)
I20260812 06:56:11.341131 19339 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:11.349448 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19339
I20260812 06:56:11.349613 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:11.350167 17685 external_mini_cluster.cc:1428] Setting key d6e50f0c3670bcfaf8da33d6d339a2c9
I20260812 06:56:11.352506 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.343428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:56217 (local address 127.17.69.126:34795)
0812 06:56:11.343598 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.343603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.344839 (+ 1236us) server_negotiation.cc:407] Connection header received
0812 06:56:11.346510 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.346515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.346582 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.346665 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.347585 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.348256 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.349696 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.350318 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.350465 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.351788 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.351795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.351799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:11.351820 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.351860 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:11.351876 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.351979 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.352191 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.352264 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:11.353538 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.343857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:34795 (local address 127.17.69.67:56217)
0812 06:56:11.344699 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.344733 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.346253 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.346839 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.346846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.346922 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:11.347450 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.347466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.348379 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.348383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.349542 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.349556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.350525 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.350530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.350794 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.351499 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:11.351504 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:11.351542 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:11.351613 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:11.351979 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:11.351988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:11.351995 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.352190 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.352546 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":277,"threads_started":1}
I20260812 06:56:11.354686 19561 heartbeater.cc:347] Connected to a master server at 127.17.69.126:34795
I20260812 06:56:11.354945 19561 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:11.355615 19561 heartbeater.cc:511] Master 127.17.69.126:34795 requested a full tablet report, sending...
I20260812 06:56:11.356835 18791 ts_manager.cc:194] Registered new tserver with Master: f8cd079999b54ba497c9e55003284862 (127.17.69.67:38013)
I20260812 06:56:11.357595 18791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.67:56217
I20260812 06:56:11.364953 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:11.374300 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.368884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.17.69.126:34795)
0812 06:56:11.369071 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.369077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.369520 (+ 443us) server_negotiation.cc:407] Connection header received
0812 06:56:11.369672 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.369675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.369733 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.369810 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.370206 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.370998 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.372259 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.372643 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.372723 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.373672 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.373677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.373680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:11.373696 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.373727 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:11.373738 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.373854 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.374024 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.374065 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:56:11.377599 18791 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:11.384379 18791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:11.415536 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.395731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.17.69.66:34009)
0812 06:56:11.406263 (+ 10532us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.406273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.407218 (+ 945us) server_negotiation.cc:407] Connection header received
0812 06:56:11.407341 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.407350 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.407600 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.408560 (+ 960us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.408799 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.410421 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.413262 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.414442 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.414560 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.414802 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.415059 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.415226 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10345,"thread_start_us":10230,"threads_started":1}
I20260812 06:56:11.415526 19586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.395569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34009 (local address 127.0.0.1:48320)
0812 06:56:11.406793 (+ 11224us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.406844 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.407026 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.407833 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.407841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.407876 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.408326 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.408342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.410564 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.410569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.413006 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.413022 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.414622 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.414629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.414845 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.414897 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.415002 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.415173 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10932,"thread_start_us":10771,"threads_started":1}
I20260812 06:56:11.419317 19226 tablet_service.cc:1467] Processing CreateTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 (DEFAULT_TABLE table=test-table [id=b346d4c74e10445da26b6223ef59f8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:11.420065 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.410710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:38013 (local address 127.0.0.1:43074)
0812 06:56:11.412322 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.412339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.412450 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.413335 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.413339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.413364 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.413689 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.413700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.415522 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.415530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.417566 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.417578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.419362 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.419367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.419491 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.419513 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.419587 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.419705 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1494,"mutex_wait_us":1,"thread_start_us":1353,"threads_started":1}
I20260812 06:56:11.420143 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.407523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.17.69.67:38013)
0812 06:56:11.412097 (+ 4574us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.412105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.412566 (+ 461us) server_negotiation.cc:407] Connection header received
0812 06:56:11.412659 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.412666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.412873 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.413046 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.413895 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.415366 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.417719 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.418875 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.419002 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.419262 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.419650 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.419800 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4392,"thread_start_us":4241,"threads_started":1}
I20260812 06:56:11.420513 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.395722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40899 (local address 127.0.0.1:41988)
0812 06:56:11.411679 (+ 15957us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.411699 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.411820 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.412816 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.412821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.412843 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.413276 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.413298 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.415198 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.415204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.417428 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.417443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.419508 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.419514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.419680 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.419711 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.419799 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.419916 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15827,"mutex_wait_us":53,"thread_start_us":15690,"threads_started":1}
I20260812 06:56:11.421068 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.396366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.17.69.65:40899)
0812 06:56:11.408869 (+ 12503us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.408878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.412175 (+ 3297us) server_negotiation.cc:407] Connection header received
0812 06:56:11.412308 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.412316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.412545 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.413424 (+ 879us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.413635 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.415038 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.417641 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.419015 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.419144 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.419364 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.420442 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.420743 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12324,"thread_start_us":12498,"threads_started":1}
I20260812 06:56:11.421100 19226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c304aeb27844d2a8bfa26665fc26d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:11.423326 19459 tablet_service.cc:1467] Processing CreateTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 (DEFAULT_TABLE table=test-table [id=b346d4c74e10445da26b6223ef59f8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:11.424736 19459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c304aeb27844d2a8bfa26665fc26d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:11.424474 19006 tablet_service.cc:1467] Processing CreateTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 (DEFAULT_TABLE table=test-table [id=b346d4c74e10445da26b6223ef59f8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:11.426247 19006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c304aeb27844d2a8bfa26665fc26d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:11.432222 19599 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Bootstrap starting.
I20260812 06:56:11.435565 19599 tablet_bootstrap.cc:654] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:11.436509 19599 log.cc:824] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:11.438478 19601 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Bootstrap starting.
I20260812 06:56:11.439332 19599 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: No bootstrap required, opened a new log
I20260812 06:56:11.439558 19599 ts_tablet_manager.cc:1397] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260812 06:56:11.442198 19601 tablet_bootstrap.cc:654] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:11.442797 19602 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Bootstrap starting.
I20260812 06:56:11.443353 19601 log.cc:824] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:11.447276 19602 tablet_bootstrap.cc:654] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:11.448585 19602 log.cc:824] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:11.449702 19599 raft_consensus.cc:348] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.450270 19599 raft_consensus.cc:374] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:11.450433 19599 raft_consensus.cc:729] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65837c1f6ebe4442ab16a592eb27fb30, State: Initialized, Role: FOLLOWER
I20260812 06:56:11.451138 19599 consensus_queue.cc:260] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.454063 19601 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: No bootstrap required, opened a new log
I20260812 06:56:11.454071 19602 tablet_bootstrap.cc:492] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: No bootstrap required, opened a new log
I20260812 06:56:11.454358 19601 ts_tablet_manager.cc:1397] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260812 06:56:11.454363 19602 ts_tablet_manager.cc:1397] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 06:56:11.455137 19609 raft_consensus.cc:482] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:11.455470 19332 heartbeater.cc:503] Master 127.17.69.126:34795 was elected leader, sending a full tablet report...
I20260812 06:56:11.456091 19599 ts_tablet_manager.cc:1428] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260812 06:56:11.456187 19609 raft_consensus.cc:504] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.462903 19601 raft_consensus.cc:348] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.463376 19601 raft_consensus.cc:374] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:11.463522 19601 raft_consensus.cc:729] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cd079999b54ba497c9e55003284862, State: Initialized, Role: FOLLOWER
I20260812 06:56:11.464025 19120 heartbeater.cc:503] Master 127.17.69.126:34795 was elected leader, sending a full tablet report...
I20260812 06:56:11.464373 19601 consensus_queue.cc:260] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.469210 19609 leader_election.cc:288] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 (127.17.69.65:40899), f8cd079999b54ba497c9e55003284862 (127.17.69.67:38013)
I20260812 06:56:11.476363 19602 raft_consensus.cc:348] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.476877 19602 raft_consensus.cc:374] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:11.477028 19602 raft_consensus.cc:729] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc9105fa51e4e0f8b5e3cb1ade9e0b2, State: Initialized, Role: FOLLOWER
I20260812 06:56:11.477762 19602 consensus_queue.cc:260] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.481067 19561 heartbeater.cc:503] Master 127.17.69.126:34795 was elected leader, sending a full tablet report...
I20260812 06:56:11.480995 19622 raft_consensus.cc:482] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:11.481374 19622 raft_consensus.cc:504] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.482762 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.458538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:46027 (local address 127.17.69.65:40899)
0812 06:56:11.458724 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.458731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.473594 (+ 14863us) server_negotiation.cc:407] Connection header received
0812 06:56:11.474064 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.474071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.474161 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.474225 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.476689 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.477695 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.480385 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.481593 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.481717 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.481785 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.482366 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.482498 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:56:11.483532 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.458063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40899 (local address 127.17.69.66:46027)
0812 06:56:11.473466 (+ 15403us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.473489 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.473892 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.474455 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.474461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.474491 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.474935 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.474946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.477878 (+ 2932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.477883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.480183 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.480198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.481958 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.481964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.482150 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.482188 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.483098 (+ 910us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.483239 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15242,"thread_start_us":17150,"threads_started":1}
I20260812 06:56:11.484777 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.463403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:41565 (local address 127.17.69.67:38013)
0812 06:56:11.463569 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.463576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.475577 (+ 12001us) server_negotiation.cc:407] Connection header received
0812 06:56:11.475788 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.475793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.475862 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.475923 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.479140 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.480204 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.482505 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.483688 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.483808 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.483871 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.484428 (+ 557us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.484549 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 06:56:11.484686 19037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "65837c1f6ebe4442ab16a592eb27fb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" is_pre_election: true
I20260812 06:56:11.485148 19037 raft_consensus.cc:2454] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65837c1f6ebe4442ab16a592eb27fb30 in term 0.
I20260812 06:56:11.485397 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.462941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:38013 (local address 127.17.69.66:41565)
0812 06:56:11.475483 (+ 12542us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.475501 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.475631 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.476112 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.476117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.476139 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.476486 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.476499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.480391 (+ 3892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.480395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.482245 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.482259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.484019 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.484025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.484253 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.484287 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.484982 (+ 695us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.485105 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12429,"thread_start_us":10672,"threads_started":1}
I20260812 06:56:11.485543 19622 leader_election.cc:288] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65837c1f6ebe4442ab16a592eb27fb30 (127.17.69.66:34009), 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 (127.17.69.65:40899)
I20260812 06:56:11.485821 19153 leader_election.cc:302] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65837c1f6ebe4442ab16a592eb27fb30, 9bc9105fa51e4e0f8b5e3cb1ade9e0b2; no voters:
I20260812 06:56:11.486148 19609 raft_consensus.cc:2781] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:11.486275 19609 raft_consensus.cc:482] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:11.486399 19609 raft_consensus.cc:3037] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:11.486310 19492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "65837c1f6ebe4442ab16a592eb27fb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cd079999b54ba497c9e55003284862" is_pre_election: true
I20260812 06:56:11.486750 19492 raft_consensus.cc:2454] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65837c1f6ebe4442ab16a592eb27fb30 in term 0.
I20260812 06:56:11.488314 19601 ts_tablet_manager.cc:1428] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Time spent starting tablet: real 0.034s user 0.008s sys 0.007s
I20260812 06:56:11.489408 19609 raft_consensus.cc:504] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.489928 19602 ts_tablet_manager.cc:1428] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Time spent starting tablet: real 0.035s user 0.010s sys 0.007s
I20260812 06:56:11.490483 19492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "65837c1f6ebe4442ab16a592eb27fb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cd079999b54ba497c9e55003284862"
I20260812 06:56:11.490700 19492 raft_consensus.cc:3037] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:11.492511 19609 leader_election.cc:288] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [CANDIDATE]: Term 1 election: Requested vote from peers 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 (127.17.69.65:40899), f8cd079999b54ba497c9e55003284862 (127.17.69.67:38013)
I20260812 06:56:11.493415 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.485855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:40899 (local address 127.17.69.67:45895)
0812 06:56:11.486172 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.486190 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.486288 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.487253 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.487258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.487284 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.487435 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.487444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.488861 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.488867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.490677 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.490695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.492712 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.492718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.492905 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.492941 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.493018 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.493155 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260812 06:56:11.493878 19492 raft_consensus.cc:2454] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65837c1f6ebe4442ab16a592eb27fb30 in term 1.
I20260812 06:56:11.494103 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.486512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:45895 (local address 127.17.69.65:40899)
0812 06:56:11.486788 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.486793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.486810 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:56:11.486863 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.486867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.486962 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.487045 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.487603 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.488658 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.491252 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.492419 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.492557 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.492617 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.493754 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.493871 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260812 06:56:11.494329 19154 leader_election.cc:302] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65837c1f6ebe4442ab16a592eb27fb30, f8cd079999b54ba497c9e55003284862; no voters:
I20260812 06:56:11.494642 19035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "f8cd079999b54ba497c9e55003284862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" is_pre_election: true
I20260812 06:56:11.494813 19609 raft_consensus.cc:2781] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:11.494896 19035 raft_consensus.cc:2454] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8cd079999b54ba497c9e55003284862 in term 0.
I20260812 06:56:11.495370 19609 raft_consensus.cc:686] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 1 LEADER]: Becoming Leader. State: Replica: 65837c1f6ebe4442ab16a592eb27fb30, State: Running, Role: LEADER
I20260812 06:56:11.495375 19037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "65837c1f6ebe4442ab16a592eb27fb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2"
I20260812 06:56:11.495385 19369 leader_election.cc:302] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc9105fa51e4e0f8b5e3cb1ade9e0b2, f8cd079999b54ba497c9e55003284862; no voters:
I20260812 06:56:11.495549 19037 raft_consensus.cc:3037] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:11.495898 19620 raft_consensus.cc:2741] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 06:56:11.495982 19609 consensus_queue.cc:237] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } }
I20260812 06:56:11.498517 19037 raft_consensus.cc:2454] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65837c1f6ebe4442ab16a592eb27fb30 in term 1.
I20260812 06:56:11.498667 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.490605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:34009 (local address 127.17.69.67:44853)
0812 06:56:11.492110 (+ 1505us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:11.492127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:11.492237 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:11.493599 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:11.493603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:11.493627 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:11.493976 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.493988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.495316 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.495320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.496937 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:11.496949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.498014 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.498021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:11.498170 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.498191 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 06:56:11.498263 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 06:56:11.498397 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":1200,"threads_started":1}
I20260812 06:56:11.498661 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.493084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:44853 (local address 127.17.69.66:34009)
0812 06:56:11.493216 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.493221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.493237 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:56:11.493287 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.493291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.493365 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.493443 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:11.494172 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.495138 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.497115 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.497870 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.498018 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.498081 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.498303 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.498439 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 06:56:11.499629 19266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c304aeb27844d2a8bfa26665fc26d12" candidate_uuid: "f8cd079999b54ba497c9e55003284862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65837c1f6ebe4442ab16a592eb27fb30" is_pre_election: true
I20260812 06:56:11.501313 18789 catalog_manager.cc:5495] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65837c1f6ebe4442ab16a592eb27fb30 (127.17.69.66). New cstate: current_term: 1 leader_uuid: "65837c1f6ebe4442ab16a592eb27fb30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65837c1f6ebe4442ab16a592eb27fb30" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 34009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 } health_report { overall_health: UNKNOWN } } }
I20260812 06:56:11.529270 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.524633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.17.69.66:34009)
0812 06:56:11.524767 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.524771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.524788 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:56:11.524945 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.524948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.525027 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.525135 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:11.525426 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.526111 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.527533 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.527938 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.528027 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.528416 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:11.528846 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:11.528973 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.529050 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.529084 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 06:56:11.536342 19492 raft_consensus.cc:1261] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 1 FOLLOWER]: Refusing update from remote peer 65837c1f6ebe4442ab16a592eb27fb30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:11.536367 19037 raft_consensus.cc:1261] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 1 FOLLOWER]: Refusing update from remote peer 65837c1f6ebe4442ab16a592eb27fb30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:11.537176 19607 consensus_queue.cc:1035] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc9105fa51e4e0f8b5e3cb1ade9e0b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 40899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:11.537477 19609 consensus_queue.cc:1035] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cd079999b54ba497c9e55003284862" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 38013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:11.547869 19634 mvcc.cc:205] Tried to move back new op lower bound from 7317576792203739136 to 7317576792052355072. Current Snapshot: MvccSnapshot[applied={T|T < 7317576792203739136}]
I20260812 06:56:11.559914 18791 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:11.560137 18791 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:11.563184 18791 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8be90f6166c64cec8c8ea701a62b02bb: Sending DeleteTablet for 3 replicas of tablet 1c304aeb27844d2a8bfa26665fc26d12
I20260812 06:56:11.564236 19459 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:11 UTC) from {username='slave'} at 127.0.0.1:43074
I20260812 06:56:11.564237 19006 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:11 UTC) from {username='slave'} at 127.0.0.1:41988
I20260812 06:56:11.564958 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18890
I20260812 06:56:11.565918 19226 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c304aeb27844d2a8bfa26665fc26d12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:11 UTC) from {username='slave'} at 127.0.0.1:48320
I20260812 06:56:11.575233 19647 tablet_replica.cc:330] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: stopping tablet replica
I20260812 06:56:11.575621 19647 raft_consensus.cc:2229] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:11.576066 19647 raft_consensus.cc:2258] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:11.576673 19650 tablet_replica.cc:330] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: stopping tablet replica
I20260812 06:56:11.576812 19648 tablet_replica.cc:330] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: stopping tablet replica
I20260812 06:56:11.577154 19648 raft_consensus.cc:2229] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:11.577142 19650 raft_consensus.cc:2229] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:11.577586 19648 raft_consensus.cc:2258] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:11.577867 19650 raft_consensus.cc:2258] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:11.579667 19650 ts_tablet_manager.cc:1905] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:11.580044 19647 ts_tablet_manager.cc:1905] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:11.583607 19648 ts_tablet_manager.cc:1905] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:11.586508 19650 ts_tablet_manager.cc:1918] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:11.586673 19650 log.cc:1196] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/1c304aeb27844d2a8bfa26665fc26d12
I20260812 06:56:11.587152 19647 ts_tablet_manager.cc:1918] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:11.587225 19650 ts_tablet_manager.cc:1939] T 1c304aeb27844d2a8bfa26665fc26d12 P 65837c1f6ebe4442ab16a592eb27fb30: Deleting consensus metadata
I20260812 06:56:11.587396 19647 log.cc:1196] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/1c304aeb27844d2a8bfa26665fc26d12
I20260812 06:56:11.587939 19647 ts_tablet_manager.cc:1939] T 1c304aeb27844d2a8bfa26665fc26d12 P f8cd079999b54ba497c9e55003284862: Deleting consensus metadata
I20260812 06:56:11.588798 18764 catalog_manager.cc:4841] TS 65837c1f6ebe4442ab16a592eb27fb30 (127.17.69.66:34009): tablet 1c304aeb27844d2a8bfa26665fc26d12 (table test-table [id=b346d4c74e10445da26b6223ef59f8f6]) successfully deleted
I20260812 06:56:11.589581 18764 catalog_manager.cc:4841] TS f8cd079999b54ba497c9e55003284862 (127.17.69.67:38013): tablet 1c304aeb27844d2a8bfa26665fc26d12 (table test-table [id=b346d4c74e10445da26b6223ef59f8f6]) successfully deleted
I20260812 06:56:11.589807 19648 ts_tablet_manager.cc:1918] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:11.589932 19648 log.cc:1196] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/1c304aeb27844d2a8bfa26665fc26d12
I20260812 06:56:11.590329 19648 ts_tablet_manager.cc:1939] T 1c304aeb27844d2a8bfa26665fc26d12 P 9bc9105fa51e4e0f8b5e3cb1ade9e0b2: Deleting consensus metadata
I20260812 06:56:11.591619 18763 catalog_manager.cc:4841] TS 9bc9105fa51e4e0f8b5e3cb1ade9e0b2 (127.17.69.65:40899): tablet 1c304aeb27844d2a8bfa26665fc26d12 (table test-table [id=b346d4c74e10445da26b6223ef59f8f6]) successfully deleted
I20260812 06:56:11.592402 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.566031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.17.69.65:40899)
0812 06:56:11.566190 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.566195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.577000 (+ 10805us) server_negotiation.cc:407] Connection header received
0812 06:56:11.577249 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.577255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.577345 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.577419 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.578458 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.579428 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.581226 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.581809 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.581907 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.590799 (+ 8892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.590811 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.590825 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:11.591576 (+ 751us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.591638 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:11.591665 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.591774 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.591949 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.592049 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:56:11.592974 19114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:11.732950 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18890
I20260812 06:56:11.761457 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19124
I20260812 06:56:11.768252 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.762203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.17.69.66:34009)
0812 06:56:11.762398 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.762404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.762488 (+ 84us) server_negotiation.cc:407] Connection header received
0812 06:56:11.762671 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.762675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.762760 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.762855 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.763249 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.764001 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.765220 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.765656 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.765777 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.766902 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.766911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.766930 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:11.767518 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.767566 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:11.767591 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.767690 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.767830 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.767939 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 06:56:11.768851 19321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:11.875306 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19124
I20260812 06:56:11.898119 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19339
I20260812 06:56:11.904174 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:11.898807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.17.69.67:38013)
0812 06:56:11.898965 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:11.898969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:11.899011 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:11.899169 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:11.899173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:11.899258 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:11.899358 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:11.899705 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.900490 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.901710 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:11.902085 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:11.902172 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:11.903039 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:11.903047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:11.903062 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:11.903624 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:11.903674 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:11.903695 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:11.903775 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:11.903858 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 06:56:11.903936 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:56:11.904779 19553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:11.991190 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19339
I20260812 06:56:12.010846 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 18736
I20260812 06:56:12.011854 18842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:12.116036 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 18736
2026-08-12T06:56:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2673 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:12 dist-test-slave-6zbq krb5kdc[19663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:12 dist-test-slave-6zbq krb5kdc[19663](info): set up 2 sockets
Aug 12 06:56:12 dist-test-slave-6zbq krb5kdc[19663](info): commencing operation
krb5kdc: starting...
W20260812 06:56:13.360298 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:13 dist-test-slave-6zbq krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:13.601470 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36515
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:13.912344 19692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:13.912573 19692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:13.912667 19692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:13.919420 19692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:13.919503 19692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:13.919555 19692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:13.919610 19692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:13.919651 19692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:13.928654 19692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:36515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:13.929630 19692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:13.930729 19692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:13.941510 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:13.941596 19708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:13.941604 19714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:13.942358 19692 server_base.cc:1034] running on GCE node
I20260812 06:56:13.943023 19692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:13.944223 19692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:13.945406 19692 hybrid_clock.cc:648] HybridClock initialized: now 1786517773945355 us; error 43 us; skew 500 ppm
I20260812 06:56:13.945708 19692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:13 dist-test-slave-6zbq krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:13.950867 19692 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:13.952175 19692 webserver.cc:460] Webserver started at http://127.17.69.126:41443/ using document root <none> and password file <none>
I20260812 06:56:13.952736 19692 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:13.952832 19692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:13.953059 19692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:13.955721 19692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "e02e9d00048140a8a4499a9e457ebf4e"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-6zbq"
server_key: "97b854ed9e9c695838b89afbd5b8756a"
server_key_iv: "43f061394994c347c91fb4e963748b84"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.956363 19692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "e02e9d00048140a8a4499a9e457ebf4e"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-6zbq"
server_key: "97b854ed9e9c695838b89afbd5b8756a"
server_key_iv: "43f061394994c347c91fb4e963748b84"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.961006 19692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 06:56:13.964098 19723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:13.965235 19692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:56:13.965373 19692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "e02e9d00048140a8a4499a9e457ebf4e"
format_stamp: "Formatted at 2026-08-12 06:56:13 on dist-test-slave-6zbq"
server_key: "97b854ed9e9c695838b89afbd5b8756a"
server_key_iv: "43f061394994c347c91fb4e963748b84"
server_key_version: "encryptionkey@0"
I20260812 06:56:13.965538 19692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:13.994416 19692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:13.998238 19692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:13.998520 19692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:14.015312 19692 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:36515
I20260812 06:56:14.015334 19821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:36515 every 8 connection(s)
I20260812 06:56:14.016538 19692 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:14.019833 19822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:14.022320 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19692
I20260812 06:56:14.022519 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:14.022943 17685 external_mini_cluster.cc:1428] Setting key bd927ec7b4b643721292b0d1ff925f40
I20260812 06:56:14.030882 19822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: Bootstrap starting.
I20260812 06:56:14.033650 19822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:14.034595 19822 log.cc:824] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:56:14 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:14.037030 19822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: No bootstrap required, opened a new log
I20260812 06:56:14.043681 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:14.024643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45180 (local address 127.17.69.126:36515)
0812 06:56:14.025433 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:14.025444 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:14.025482 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:56:14.027132 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:14.027153 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:14.027512 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:14.027806 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:14.029617 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.030923 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.032211 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.032863 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.033051 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:14.036843 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:14.036850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:14.036863 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:14.036902 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:14.039893 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:14.040536 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:14.040543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:14.040551 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:14.040724 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:14.041105 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:14.041109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:14.041111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:14.041631 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:14.041933 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:14.042401 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 06:56:14.042771 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":362,"threads_started":1}
I20260812 06:56:14.044421 19822 raft_consensus.cc:348] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } }
I20260812 06:56:14.044739 19822 raft_consensus.cc:374] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:14.044919 19822 raft_consensus.cc:729] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02e9d00048140a8a4499a9e457ebf4e, State: Initialized, Role: FOLLOWER
I20260812 06:56:14.045442 19822 consensus_queue.cc:260] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } }
I20260812 06:56:14.045676 19822 raft_consensus.cc:388] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:14.045812 19822 raft_consensus.cc:482] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:14.045964 19822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:14.047931 19822 raft_consensus.cc:504] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } }
I20260812 06:56:14.048360 19822 leader_election.cc:302] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e02e9d00048140a8a4499a9e457ebf4e; no voters:
I20260812 06:56:14.048874 19822 leader_election.cc:288] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:14.048986 19833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:14.049917 19833 raft_consensus.cc:686] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [term 1 LEADER]: Becoming Leader. State: Replica: e02e9d00048140a8a4499a9e457ebf4e, State: Running, Role: LEADER
I20260812 06:56:14.050681 19822 sys_catalog.cc:564] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:14.050670 19833 consensus_queue.cc:237] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } }
I20260812 06:56:14.053292 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e02e9d00048140a8a4499a9e457ebf4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } } }
I20260812 06:56:14.053515 19837 sys_catalog.cc:455] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e02e9d00048140a8a4499a9e457ebf4e. Latest consensus state: current_term: 1 leader_uuid: "e02e9d00048140a8a4499a9e457ebf4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e02e9d00048140a8a4499a9e457ebf4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 36515 } } }
I20260812 06:56:14.053697 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:14.053787 19837 sys_catalog.cc:458] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:14.054817 19848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:14.065610 19848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:14.072716 19848 catalog_manager.cc:1347] Generated new cluster ID: e76a9cca3fe44444a62c4dfc452ad9e3
I20260812 06:56:14.072824 19848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:14.084844 19848 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:14.086517 19848 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:14.102831 19848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: Generated new TSK 0
I20260812 06:56:14.103659 19848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.69.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
Entry for principal kudu/127.17.69.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab.
I20260812 06:56:14.171062 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:14.486629 19871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:14.486861 19871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:14.486989 19871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:14.493808 19871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:14.493906 19871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:14.494059 19871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:14.503049 19871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.65.keytab
--principal=kudu/127.17.69.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:14.504065 19871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:14.505201 19871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:14.516925 19882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:14.517047 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:14.516959 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:14.517851 19871 server_base.cc:1034] running on GCE node
I20260812 06:56:14.518486 19871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:14.519308 19871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:14.520507 19871 hybrid_clock.cc:648] HybridClock initialized: now 1786517774520461 us; error 35 us; skew 500 ppm
I20260812 06:56:14.520793 19871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:14 dist-test-slave-6zbq krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:14.523756 19871 init.cc:377] Logged in from keytab as kudu/127.17.69.65@KRBTEST.COM (short username kudu)
I20260812 06:56:14.525008 19871 webserver.cc:460] Webserver started at http://127.17.69.65:32889/ using document root <none> and password file <none>
I20260812 06:56:14.525621 19871 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:14.525738 19871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:14.526014 19871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:14.528702 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "023f00478f9549d0890480e7d748e4b2"
format_stamp: "Formatted at 2026-08-12 06:56:14 on dist-test-slave-6zbq"
server_key: "8bcd3c410da76bfcec680ab67f496447"
server_key_iv: "6cab435e664ae3e9665ee39108e893b5"
server_key_version: "encryptionkey@0"
I20260812 06:56:14.529419 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "023f00478f9549d0890480e7d748e4b2"
format_stamp: "Formatted at 2026-08-12 06:56:14 on dist-test-slave-6zbq"
server_key: "8bcd3c410da76bfcec680ab67f496447"
server_key_iv: "6cab435e664ae3e9665ee39108e893b5"
server_key_version: "encryptionkey@0"
I20260812 06:56:14.534169 19871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 06:56:14.537575 19895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:14.538805 19871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:56:14.538954 19871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "023f00478f9549d0890480e7d748e4b2"
format_stamp: "Formatted at 2026-08-12 06:56:14 on dist-test-slave-6zbq"
server_key: "8bcd3c410da76bfcec680ab67f496447"
server_key_iv: "6cab435e664ae3e9665ee39108e893b5"
server_key_version: "encryptionkey@0"
I20260812 06:56:14.539129 19871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:14.551328 19871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:14.555215 19871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:14.555454 19871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:14.556267 19871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:14.557428 19871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:14.557503 19871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:14.557614 19871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:14.557667 19871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:14.586578 19871 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:41903
I20260812 06:56:14.586629 20100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:41903 every 8 connection(s)
I20260812 06:56:14.587659 19871 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:14.594400 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 19871
I20260812 06:56:14.594578 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:14.594959 17685 external_mini_cluster.cc:1428] Setting key a1e7166b278d41d6c642209c55634e6d
Aug 12 06:56:14 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.65@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:14.605506 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:14.589739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:47909 (local address 127.17.69.126:36515)
0812 06:56:14.590122 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:14.590126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:14.590926 (+ 800us) server_negotiation.cc:407] Connection header received
0812 06:56:14.592627 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:14.592633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:14.592709 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:14.592811 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:14.594148 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.594947 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.596522 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.596940 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.597046 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:14.600130 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:14.600135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:14.600138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:14.600170 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:14.602475 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:14.603111 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:14.603117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:14.603120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:14.603258 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:14.603659 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:14.603664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:14.603668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:14.604002 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:14.604141 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:14.604724 (+ 583us) server_negotiation.cc:299] Negotiation successful
0812 06:56:14.605145 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":202,"threads_started":1}
I20260812 06:56:14.606143 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:14.590120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36515 (local address 127.17.69.65:47909)
0812 06:56:14.590773 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:14.590818 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:14.592385 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:14.593027 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:14.593034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:14.593497 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:14.593987 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:14.594004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.595102 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.595105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:14.596383 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:14.596397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:14.597142 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:14.597150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:14.597365 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:14.597917 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:14.597937 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:14.599938 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:14.602588 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:14.602597 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:14.602611 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:14.603003 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:14.603373 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:14.603378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:14.603382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:14.603554 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:14.604125 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:14.604132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:14.604404 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 06:56:14.604621 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 06:56:14.605075 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":270,"threads_started":1}
I20260812 06:56:14.608520 20102 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36515
I20260812 06:56:14.608860 20102 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:14.609639 20102 heartbeater.cc:511] Master 127.17.69.126:36515 requested a full tablet report, sending...
I20260812 06:56:14.611922 19759 ts_manager.cc:194] Registered new tserver with Master: 023f00478f9549d0890480e7d748e4b2 (127.17.69.65:41903)
I20260812 06:56:14.613293 19759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.65@KRBTEST.COM'} at 127.17.69.65:47909
WARNING: no policy specified for kudu/127.17.69.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
Entry for principal kudu/127.17.69.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab.
I20260812 06:56:14.657153 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:14.973105 20112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:14.973344 20112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:14.973460 20112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:14.980386 20112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:14.980479 20112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:14.980621 20112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:14.989898 20112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.66.keytab
--principal=kudu/127.17.69.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:14.991036 20112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:14.992247 20112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:15.004084 20122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.004055 20123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.004204 20130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:15.004684 20112 server_base.cc:1034] running on GCE node
I20260812 06:56:15.005195 20112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:15.005981 20112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:15.007177 20112 hybrid_clock.cc:648] HybridClock initialized: now 1786517775007132 us; error 39 us; skew 500 ppm
I20260812 06:56:15.007437 20112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:15.010267 20112 init.cc:377] Logged in from keytab as kudu/127.17.69.66@KRBTEST.COM (short username kudu)
I20260812 06:56:15.011590 20112 webserver.cc:460] Webserver started at http://127.17.69.66:39815/ using document root <none> and password file <none>
I20260812 06:56:15.012178 20112 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:15.012274 20112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:15.012511 20112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:15.015504 20112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "5ba50b8c46c84ae88d48353c0679c70c"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "8da8470f94faa7db36b9489a131f19c3"
server_key_iv: "b73b7a6f8a1b34a360fadb28c1e696ad"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.016433 20112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "5ba50b8c46c84ae88d48353c0679c70c"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "8da8470f94faa7db36b9489a131f19c3"
server_key_iv: "b73b7a6f8a1b34a360fadb28c1e696ad"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.022336 20112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 06:56:15.026238 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.027704 20112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 06:56:15.027855 20112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "5ba50b8c46c84ae88d48353c0679c70c"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "8da8470f94faa7db36b9489a131f19c3"
server_key_iv: "b73b7a6f8a1b34a360fadb28c1e696ad"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.028052 20112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:15.041726 20112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:15.045356 20112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:15.045575 20112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:15.046408 20112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:15.047703 20112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:15.047771 20112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.047857 20112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:15.047899 20112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.078500 20112 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:43875
I20260812 06:56:15.078537 20346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:43875 every 8 connection(s)
I20260812 06:56:15.079622 20112 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:15.082798 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20112
I20260812 06:56:15.082999 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:15.083494 17685 external_mini_cluster.cc:1428] Setting key a7826d25bed08df11c9362b0393533e9
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.66@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:15.098749 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.081849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:51421 (local address 127.17.69.126:36515)
0812 06:56:15.082020 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.082024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.083071 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 06:56:15.085035 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.085039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.085104 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.085201 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.086707 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.087406 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.088880 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.089297 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.089402 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.092572 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.092577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.092580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.092612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.095955 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.096614 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.096620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.096624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.096764 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.097166 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.097170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.097172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.097410 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.097548 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.098183 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.098436 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 06:56:15.099563 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.082200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36515 (local address 127.17.69.66:51421)
0812 06:56:15.082907 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.082962 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.084772 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.085461 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.085472 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.085978 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:15.086540 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.086559 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.087554 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.087558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.088743 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.088756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.089507 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.089515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.089732 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.090323 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:15.090379 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:15.092376 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:15.096080 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:15.096088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:15.096102 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:15.096492 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:15.096867 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:15.096872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:15.096876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:15.097043 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:15.097531 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:15.097538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:15.097832 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.098043 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.098523 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":312,"threads_started":1}
I20260812 06:56:15.100829 20347 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36515
I20260812 06:56:15.101141 20347 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:15.101953 20347 heartbeater.cc:511] Master 127.17.69.126:36515 requested a full tablet report, sending...
I20260812 06:56:15.103389 19759 ts_manager.cc:194] Registered new tserver with Master: 5ba50b8c46c84ae88d48353c0679c70c (127.17.69.66:43875)
I20260812 06:56:15.104125 19759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.66@KRBTEST.COM'} at 127.17.69.66:51421
WARNING: no policy specified for kudu/127.17.69.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
Entry for principal kudu/127.17.69.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab.
I20260812 06:56:15.144989 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:15.454934 20357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:15.455170 20357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:15.455268 20357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:15.462003 20357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:15.462088 20357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:15.462200 20357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:15.471083 20357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.67.keytab
--principal=kudu/127.17.69.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:36515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:15.472088 20357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:15.473232 20357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:15.484957 20363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.484992 20366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:15.485023 20372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:15.485806 20357 server_base.cc:1034] running on GCE node
I20260812 06:56:15.486308 20357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:15.487152 20357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:15.488344 20357 hybrid_clock.cc:648] HybridClock initialized: now 1786517775488298 us; error 40 us; skew 500 ppm
I20260812 06:56:15.488618 20357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:15.491464 20357 init.cc:377] Logged in from keytab as kudu/127.17.69.67@KRBTEST.COM (short username kudu)
I20260812 06:56:15.492738 20357 webserver.cc:460] Webserver started at http://127.17.69.67:39663/ using document root <none> and password file <none>
I20260812 06:56:15.493319 20357 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:15.493430 20357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:15.493712 20357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:15.496547 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "0e0268f1a064450d9146df76e1146b96"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "2ee6c84cdb11d6a3222f6ce04c76ef16"
server_key_iv: "14df7c593275474221f307c0818057fc"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.497257 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "0e0268f1a064450d9146df76e1146b96"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "2ee6c84cdb11d6a3222f6ce04c76ef16"
server_key_iv: "14df7c593275474221f307c0818057fc"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.501911 20357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260812 06:56:15.505025 20384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.506189 20357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:56:15.506330 20357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "0e0268f1a064450d9146df76e1146b96"
format_stamp: "Formatted at 2026-08-12 06:56:15 on dist-test-slave-6zbq"
server_key: "2ee6c84cdb11d6a3222f6ce04c76ef16"
server_key_iv: "14df7c593275474221f307c0818057fc"
server_key_version: "encryptionkey@0"
I20260812 06:56:15.506533 20357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:15.522230 20357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:15.526022 20357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:15.526273 20357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:15.527161 20357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:15.528338 20357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:15.528414 20357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.528527 20357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:15.528577 20357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:15.557324 20357 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:32989
I20260812 06:56:15.557375 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:32989 every 8 connection(s)
I20260812 06:56:15.558606 20357 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:15.567169 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20357
I20260812 06:56:15.567365 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:15.567867 17685 external_mini_cluster.cc:1428] Setting key 04cce266f13bfc89080546ca665cc53c
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.67@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:15.576128 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.560835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:35765 (local address 127.17.69.126:36515)
0812 06:56:15.561008 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.561012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.562055 (+ 1043us) server_negotiation.cc:407] Connection header received
0812 06:56:15.563748 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.563753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.563821 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.563907 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.565165 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.565911 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.567642 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.568107 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.568226 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.571615 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.571620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.571623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.571654 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.573797 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.574432 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.574436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.574438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.574548 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.574884 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.574886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.574888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.575126 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.575229 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.575737 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.575902 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:56:15.576932 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.561283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:36515 (local address 127.17.69.67:35765)
0812 06:56:15.561911 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.561944 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.563501 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.564076 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.564083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.564540 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:15.565029 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.565045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.566030 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.566033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.567506 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.567522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.568325 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.568330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.568568 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.569249 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:15.569271 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:15.571436 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:15.573916 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:15.573922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:15.573931 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:15.574320 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:15.574623 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:15.574627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:15.574631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:15.574803 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:15.575212 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:15.575217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:15.575468 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.575662 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.575979 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":257,"threads_started":1}
I20260812 06:56:15.578068 20580 heartbeater.cc:347] Connected to a master server at 127.17.69.126:36515
I20260812 06:56:15.578297 20580 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:15.578919 20580 heartbeater.cc:511] Master 127.17.69.126:36515 requested a full tablet report, sending...
I20260812 06:56:15.580101 19759 ts_manager.cc:194] Registered new tserver with Master: 0e0268f1a064450d9146df76e1146b96 (127.17.69.67:32989)
I20260812 06:56:15.580879 19759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.69.67@KRBTEST.COM'} at 127.17.69.67:35765
I20260812 06:56:15.582549 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:15.599162 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.586415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.17.69.126:36515)
0812 06:56:15.586589 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.586596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.587019 (+ 423us) server_negotiation.cc:407] Connection header received
0812 06:56:15.587173 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.587177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.587232 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.587304 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.589110 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.589862 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.591651 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.592043 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.592146 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.594587 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.594591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.594595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.594624 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.596991 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.597568 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.597572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.597574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.597706 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.597974 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.597977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.597979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.598246 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.598467 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.598720 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.598875 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:15.602708 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:15.609237 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:15.615813 20102 heartbeater.cc:503] Master 127.17.69.126:36515 was elected leader, sending a full tablet report...
I20260812 06:56:15.647660 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.620453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.17.69.65:41903)
0812 06:56:15.639230 (+ 18777us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.639237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.639265 (+ 28us) server_negotiation.cc:407] Connection header received
0812 06:56:15.639353 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.639360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.639588 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.640470 (+ 882us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:15.640722 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.642641 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.645476 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.646617 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.646735 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.646956 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.647202 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.647354 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18628,"thread_start_us":18490,"threads_started":1}
I20260812 06:56:15.647743 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.620177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:41903 (local address 127.0.0.1:60744)
0812 06:56:15.637329 (+ 17152us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.637373 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.637536 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.639821 (+ 2285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.639827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.639860 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:15.640278 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.640292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.642799 (+ 2507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.642804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.645305 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.645325 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.646801 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.646807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.646991 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.647045 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.647148 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.647327 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16923,"mutex_wait_us":36,"thread_start_us":19067,"threads_started":1}
I20260812 06:56:15.648209 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.620787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:32989 (local address 127.0.0.1:38678)
0812 06:56:15.639469 (+ 18682us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.639490 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.639591 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.640815 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.640820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.640850 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:15.641442 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.641453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.643192 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.643197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.645367 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.645377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.647100 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.647107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.647288 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.647320 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.647412 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.647534 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18574,"mutex_wait_us":33,"thread_start_us":16909,"threads_started":1}
I20260812 06:56:15.648640 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.622594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:43875 (local address 127.0.0.1:44588)
0812 06:56:15.641518 (+ 18924us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.641539 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.641651 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.642548 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.642552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.642576 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:15.642984 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.643000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.644340 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.644345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.645950 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.645964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.647915 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.647921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.648088 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.648119 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.648222 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.648338 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18781,"thread_start_us":18601,"threads_started":1}
I20260812 06:56:15.649024 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.620749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.17.69.67:32989)
0812 06:56:15.640272 (+ 19523us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.640279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.640311 (+ 32us) server_negotiation.cc:407] Connection header received
0812 06:56:15.640401 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.640409 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.640608 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.640807 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:15.641647 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.643072 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.645517 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.646632 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.646746 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.646952 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.648551 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.648700 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19354,"thread_start_us":19195,"threads_started":1}
I20260812 06:56:15.649274 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.622689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.17.69.66:43875)
0812 06:56:15.638770 (+ 16081us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.638786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.641769 (+ 2983us) server_negotiation.cc:407] Connection header received
0812 06:56:15.641861 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.641867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.642055 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.642299 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:15.643163 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.644211 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.646111 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.647472 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.647586 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.647779 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.648853 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.649014 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15918,"thread_start_us":15823,"threads_started":1}
I20260812 06:56:15.651547 19993 tablet_service.cc:1467] Processing CreateTablet for tablet 8959fa023429472588c2e92c30529ac1 (DEFAULT_TABLE table=test-table [id=1aaae1af2768437995daa0b53ae3b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:15.652293 20224 tablet_service.cc:1467] Processing CreateTablet for tablet 8959fa023429472588c2e92c30529ac1 (DEFAULT_TABLE table=test-table [id=1aaae1af2768437995daa0b53ae3b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:15.652458 20472 tablet_service.cc:1467] Processing CreateTablet for tablet 8959fa023429472588c2e92c30529ac1 (DEFAULT_TABLE table=test-table [id=1aaae1af2768437995daa0b53ae3b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:15.652995 19993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8959fa023429472588c2e92c30529ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:15.653254 20224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8959fa023429472588c2e92c30529ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:15.653755 20472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8959fa023429472588c2e92c30529ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:15.663940 20616 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Bootstrap starting.
I20260812 06:56:15.666906 20617 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Bootstrap starting.
I20260812 06:56:15.667254 20616 tablet_bootstrap.cc:654] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:15.669271 20616 log.cc:824] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:15.670354 20617 tablet_bootstrap.cc:654] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:15.670811 20618 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Bootstrap starting.
I20260812 06:56:15.671883 20617 log.cc:824] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:15.674000 20618 tablet_bootstrap.cc:654] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:15.674211 20617 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: No bootstrap required, opened a new log
I20260812 06:56:15.674387 20616 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: No bootstrap required, opened a new log
I20260812 06:56:15.674489 20617 ts_tablet_manager.cc:1397] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260812 06:56:15.674644 20616 ts_tablet_manager.cc:1397] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260812 06:56:15.675176 20618 log.cc:824] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:15.677932 20618 tablet_bootstrap.cc:492] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: No bootstrap required, opened a new log
I20260812 06:56:15.678169 20618 ts_tablet_manager.cc:1397] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 06:56:15.682250 20617 raft_consensus.cc:348] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.682603 20617 raft_consensus.cc:374] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:15.682698 20617 raft_consensus.cc:729] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 023f00478f9549d0890480e7d748e4b2, State: Initialized, Role: FOLLOWER
I20260812 06:56:15.683243 20617 consensus_queue.cc:260] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.684262 20616 raft_consensus.cc:348] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.684677 20616 raft_consensus.cc:374] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:15.684780 20616 raft_consensus.cc:729] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba50b8c46c84ae88d48353c0679c70c, State: Initialized, Role: FOLLOWER
I20260812 06:56:15.685004 20617 ts_tablet_manager.cc:1428] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 06:56:15.685356 20616 consensus_queue.cc:260] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.686549 20347 heartbeater.cc:503] Master 127.17.69.126:36515 was elected leader, sending a full tablet report...
I20260812 06:56:15.687048 20616 ts_tablet_manager.cc:1428] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260812 06:56:15.692643 20618 raft_consensus.cc:348] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.692950 20618 raft_consensus.cc:374] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:15.693048 20618 raft_consensus.cc:729] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0268f1a064450d9146df76e1146b96, State: Initialized, Role: FOLLOWER
I20260812 06:56:15.693542 20618 consensus_queue.cc:260] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.694383 20580 heartbeater.cc:503] Master 127.17.69.126:36515 was elected leader, sending a full tablet report...
I20260812 06:56:15.694780 20618 ts_tablet_manager.cc:1428] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Time spent starting tablet: real 0.017s user 0.013s sys 0.001s
I20260812 06:56:15.731704 20628 raft_consensus.cc:482] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:15.731927 20628 raft_consensus.cc:504] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.733399 20628 leader_election.cc:288] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 023f00478f9549d0890480e7d748e4b2 (127.17.69.65:41903), 0e0268f1a064450d9146df76e1146b96 (127.17.69.67:32989)
I20260812 06:56:15.739902 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.733666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:38853 (local address 127.17.69.65:41903)
0812 06:56:15.733871 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.733877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.734653 (+ 776us) server_negotiation.cc:407] Connection header received
0812 06:56:15.734833 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.734837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.734891 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.734961 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:15.735531 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.736300 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.738169 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.738949 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.739088 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.739159 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.739600 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.739722 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:56:15.739918 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.733839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:47483 (local address 127.17.69.67:32989)
0812 06:56:15.734011 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.734018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.734691 (+ 673us) server_negotiation.cc:407] Connection header received
0812 06:56:15.734858 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.734862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.734915 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.734985 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:15.735644 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.736415 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.738194 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.738981 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.739122 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.739192 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.739555 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.739684 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:56:15.740603 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.733713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:41903 (local address 127.17.69.66:38853)
0812 06:56:15.734550 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.734568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.734701 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.734984 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.734988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.735010 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:15.735400 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.735411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.736416 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.736421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.738033 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.738045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.739165 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.739172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.739417 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.739450 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.740104 (+ 654us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.740255 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":162,"thread_start_us":411,"threads_started":1}
I20260812 06:56:15.740618 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.733815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:32989 (local address 127.17.69.66:47483)
0812 06:56:15.734595 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:15.734610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:15.734739 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:15.735013 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:15.735017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:15.735040 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:15.735478 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.735488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.736549 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.736553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.738041 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:15.738054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.739202 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.739209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:15.739367 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.739393 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 06:56:15.740129 (+ 736us) client_negotiation.cc:240] Negotiation successful
0812 06:56:15.740306 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":30,"thread_start_us":434,"threads_started":1}
I20260812 06:56:15.741508 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8959fa023429472588c2e92c30529ac1" candidate_uuid: "5ba50b8c46c84ae88d48353c0679c70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "023f00478f9549d0890480e7d748e4b2" is_pre_election: true
I20260812 06:56:15.741636 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8959fa023429472588c2e92c30529ac1" candidate_uuid: "5ba50b8c46c84ae88d48353c0679c70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0268f1a064450d9146df76e1146b96" is_pre_election: true
I20260812 06:56:15.741962 20029 raft_consensus.cc:2454] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ba50b8c46c84ae88d48353c0679c70c in term 0.
I20260812 06:56:15.742063 20511 raft_consensus.cc:2454] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ba50b8c46c84ae88d48353c0679c70c in term 0.
I20260812 06:56:15.742619 20146 leader_election.cc:302] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 023f00478f9549d0890480e7d748e4b2, 5ba50b8c46c84ae88d48353c0679c70c; no voters:
I20260812 06:56:15.742964 20628 raft_consensus.cc:2781] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:15.743080 20628 raft_consensus.cc:482] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:15.743172 20628 raft_consensus.cc:3037] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:15.745255 20628 raft_consensus.cc:504] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.745801 20628 leader_election.cc:288] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [CANDIDATE]: Term 1 election: Requested vote from peers 023f00478f9549d0890480e7d748e4b2 (127.17.69.65:41903), 0e0268f1a064450d9146df76e1146b96 (127.17.69.67:32989)
I20260812 06:56:15.746133 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8959fa023429472588c2e92c30529ac1" candidate_uuid: "5ba50b8c46c84ae88d48353c0679c70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "023f00478f9549d0890480e7d748e4b2"
I20260812 06:56:15.746213 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8959fa023429472588c2e92c30529ac1" candidate_uuid: "5ba50b8c46c84ae88d48353c0679c70c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0268f1a064450d9146df76e1146b96"
I20260812 06:56:15.746292 20029 raft_consensus.cc:3037] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:15.746407 20511 raft_consensus.cc:3037] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:15.748435 20511 raft_consensus.cc:2454] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ba50b8c46c84ae88d48353c0679c70c in term 1.
I20260812 06:56:15.748435 20029 raft_consensus.cc:2454] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ba50b8c46c84ae88d48353c0679c70c in term 1.
I20260812 06:56:15.748771 20146 leader_election.cc:302] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 023f00478f9549d0890480e7d748e4b2, 0e0268f1a064450d9146df76e1146b96, 5ba50b8c46c84ae88d48353c0679c70c; no voters:
I20260812 06:56:15.749073 20628 raft_consensus.cc:2781] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:15.749466 20628 raft_consensus.cc:686] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 1 LEADER]: Becoming Leader. State: Replica: 5ba50b8c46c84ae88d48353c0679c70c, State: Running, Role: LEADER
I20260812 06:56:15.749871 20628 consensus_queue.cc:237] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } }
I20260812 06:56:15.754499 19756 catalog_manager.cc:5495] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ba50b8c46c84ae88d48353c0679c70c (127.17.69.66). New cstate: current_term: 1 leader_uuid: "5ba50b8c46c84ae88d48353c0679c70c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ba50b8c46c84ae88d48353c0679c70c" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 43875 } health_report { overall_health: HEALTHY } } }
I20260812 06:56:15.780362 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.775548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.17.69.66:43875)
0812 06:56:15.775757 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.775763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.775814 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:56:15.775992 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.775996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.776076 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.776204 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:15.776552 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.777265 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.778563 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.778955 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.779043 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.779427 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:15.779868 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:15.779965 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.780021 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.780082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:15.787840 20511 raft_consensus.cc:1261] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 1 FOLLOWER]: Refusing update from remote peer 5ba50b8c46c84ae88d48353c0679c70c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:15.787839 20029 raft_consensus.cc:1261] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 1 FOLLOWER]: Refusing update from remote peer 5ba50b8c46c84ae88d48353c0679c70c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:15.788720 20628 consensus_queue.cc:1035] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0268f1a064450d9146df76e1146b96" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:15.789011 20634 consensus_queue.cc:1035] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [LEADER]: Connected to new peer: Peer: permanent_uuid: "023f00478f9549d0890480e7d748e4b2" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 41903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:15.811100 19759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:15.811324 19759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:15.814671 19759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e02e9d00048140a8a4499a9e457ebf4e: Sending DeleteTablet for 3 replicas of tablet 8959fa023429472588c2e92c30529ac1
I20260812 06:56:15.815707 20472 tablet_service.cc:1514] Processing DeleteTablet for tablet 8959fa023429472588c2e92c30529ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:15 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:38678
I20260812 06:56:15.815706 19993 tablet_service.cc:1514] Processing DeleteTablet for tablet 8959fa023429472588c2e92c30529ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:15 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:60744
I20260812 06:56:15.815706 20224 tablet_service.cc:1514] Processing DeleteTablet for tablet 8959fa023429472588c2e92c30529ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:15 UTC) from {username='kudu', principal='kudu/127.17.69.126@KRBTEST.COM'} at 127.0.0.1:44588
I20260812 06:56:15.816516 20655 tablet_replica.cc:330] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: stopping tablet replica
I20260812 06:56:15.816942 20655 raft_consensus.cc:2229] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:15.817276 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19871
I20260812 06:56:15.817569 20655 raft_consensus.cc:2258] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:15.819585 20655 ts_tablet_manager.cc:1905] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:15.827018 20655 ts_tablet_manager.cc:1918] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:15.827291 20655 log.cc:1196] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/8959fa023429472588c2e92c30529ac1
I20260812 06:56:15.827880 20655 ts_tablet_manager.cc:1939] T 8959fa023429472588c2e92c30529ac1 P 0e0268f1a064450d9146df76e1146b96: Deleting consensus metadata
I20260812 06:56:15.827924 20657 tablet_replica.cc:330] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: stopping tablet replica
I20260812 06:56:15.828326 20657 raft_consensus.cc:2229] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:15.828415 20656 tablet_replica.cc:330] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: stopping tablet replica
I20260812 06:56:15.828817 20656 raft_consensus.cc:2229] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:15.828984 20657 raft_consensus.cc:2258] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:15.829509 20656 raft_consensus.cc:2258] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:15.830046 19733 catalog_manager.cc:4841] TS 0e0268f1a064450d9146df76e1146b96 (127.17.69.67:32989): tablet 8959fa023429472588c2e92c30529ac1 (table test-table [id=1aaae1af2768437995daa0b53ae3b004]) successfully deleted
I20260812 06:56:15.832115 20656 ts_tablet_manager.cc:1905] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:15.832603 20657 ts_tablet_manager.cc:1905] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.65@KRBTEST.COM
I20260812 06:56:15.838905 20656 ts_tablet_manager.cc:1918] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:15.839057 20656 log.cc:1196] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/8959fa023429472588c2e92c30529ac1
I20260812 06:56:15.839573 20656 ts_tablet_manager.cc:1939] T 8959fa023429472588c2e92c30529ac1 P 023f00478f9549d0890480e7d748e4b2: Deleting consensus metadata
I20260812 06:56:15.840083 20657 ts_tablet_manager.cc:1918] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:15.840241 20657 log.cc:1196] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/8959fa023429472588c2e92c30529ac1
I20260812 06:56:15.840765 20657 ts_tablet_manager.cc:1939] T 8959fa023429472588c2e92c30529ac1 P 5ba50b8c46c84ae88d48353c0679c70c: Deleting consensus metadata
I20260812 06:56:15.841292 19731 catalog_manager.cc:4841] TS 023f00478f9549d0890480e7d748e4b2 (127.17.69.65:41903): tablet 8959fa023429472588c2e92c30529ac1 (table test-table [id=1aaae1af2768437995daa0b53ae3b004]) successfully deleted
I20260812 06:56:15.842623 19725 catalog_manager.cc:4841] TS 5ba50b8c46c84ae88d48353c0679c70c (127.17.69.66:43875): tablet 8959fa023429472588c2e92c30529ac1 (table test-table [id=1aaae1af2768437995daa0b53ae3b004]) successfully deleted
I20260812 06:56:15.846215 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.820052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.17.69.65:41903)
0812 06:56:15.820226 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.820231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.828159 (+ 7928us) server_negotiation.cc:407] Connection header received
0812 06:56:15.828461 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.828467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.828571 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.828653 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.830636 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.831730 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.833660 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.834302 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.834419 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.840190 (+ 5771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.840202 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.840218 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.840273 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.843617 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.844253 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.844261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.844265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.844387 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.844755 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.844760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.844764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.845107 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.845385 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.845700 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.845898 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:56:15.846881 20091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:15.933764 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19871
I20260812 06:56:15.957929 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20112
Aug 12 06:56:15 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.66@KRBTEST.COM
I20260812 06:56:15.974176 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:15.958821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.17.69.66:43875)
0812 06:56:15.959013 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:15.959019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:15.959039 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:56:15.959106 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:15.959111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:15.959192 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:15.959297 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:15.960493 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.961679 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.962970 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:15.963649 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:15.963774 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:15.967691 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:15.967707 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:15.967728 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:15.967773 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:15.971507 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.972110 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.972119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.972123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.972285 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:15.972608 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:15.972620 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:15.972624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:15.973057 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:15.973395 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:15.973534 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 06:56:15.973766 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:56:15.975121 20340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:16.090824 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20112
I20260812 06:56:16.118530 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20357
Aug 12 06:56:16 dist-test-slave-6zbq krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.67@KRBTEST.COM
I20260812 06:56:16.132958 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:16.119384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.17.69.67:32989)
0812 06:56:16.119585 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:16.119591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:16.119612 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:56:16.119693 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:16.119697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:16.119764 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:16.119847 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:16.121223 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.121965 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.123324 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.123716 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.123829 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:16.128105 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:16.128120 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:16.128137 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:16.128171 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:16.130722 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:16.131314 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:16.131320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:16.131323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:16.131456 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:16.131745 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:16.131748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:16.131750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:16.132070 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:16.132291 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:16.132500 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 06:56:16.132660 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 06:56:16.133553 20570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:16.218995 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20357
I20260812 06:56:16.240863 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 19692
I20260812 06:56:16.241936 19812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:16.348299 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 19692
2026-08-12T06:56:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4236 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T06:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:16Z Disabled control of system clock
I20260812 06:56:16.438289 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46367
--webserver_interface=127.17.69.126
--webserver_port=0
--builtin_ntp_servers=127.17.69.84:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:46367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:16.763406 20670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:16.763655 20670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:16.763741 20670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:16.770753 20670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:16.770852 20670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:16.770896 20670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:16.770959 20670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:16.771003 20670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:16.779855 20670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:46367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:16.780946 20670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:16.782655 20670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:16.793443 20684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:16.793490 20680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:16.793514 20679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:16.794529 20670 server_base.cc:1034] running on GCE node
I20260812 06:56:16.795200 20670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:16.796446 20670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:16.797631 20670 hybrid_clock.cc:648] HybridClock initialized: now 1786517776797589 us; error 34 us; skew 500 ppm
I20260812 06:56:16.797957 20670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:16.800137 20670 webserver.cc:460] Webserver started at http://127.17.69.126:41579/ using document root <none> and password file <none>
I20260812 06:56:16.800767 20670 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:16.800856 20670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:16.801155 20670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:16.803926 20670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "c3af700a0e4f4fdc83f1bc851e01a62a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-6zbq"
server_key: "cf1b69cc1cd41cf42e429455d0e1920a"
server_key_iv: "a7960bd4112586f89a9567ab291ef1f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.804702 20670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "c3af700a0e4f4fdc83f1bc851e01a62a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-6zbq"
server_key: "cf1b69cc1cd41cf42e429455d0e1920a"
server_key_iv: "a7960bd4112586f89a9567ab291ef1f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.809517 20670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 06:56:16.812726 20695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:16.813891 20670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 06:56:16.814025 20670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "c3af700a0e4f4fdc83f1bc851e01a62a"
format_stamp: "Formatted at 2026-08-12 06:56:16 on dist-test-slave-6zbq"
server_key: "cf1b69cc1cd41cf42e429455d0e1920a"
server_key_iv: "a7960bd4112586f89a9567ab291ef1f1"
server_key_version: "encryptionkey@0"
I20260812 06:56:16.814209 20670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:16.838979 20670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:16.839697 20670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:16.839923 20670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:16.856400 20670 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:46367
I20260812 06:56:16.856419 20795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:46367 every 8 connection(s)
I20260812 06:56:16.857621 20670 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:16.860612 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20670
I20260812 06:56:16.860769 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:16.861198 17685 external_mini_cluster.cc:1428] Setting key e53143e636fe36de0468be7ffacbb820
I20260812 06:56:16.861234 20798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:16.875469 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:16.862958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.17.69.126:46367)
0812 06:56:16.864031 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:16.864045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:16.864098 (+ 53us) server_negotiation.cc:407] Connection header received
0812 06:56:16.865943 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:16.865976 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:16.866547 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:16.866969 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:16.867481 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.869410 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.870697 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:16.871651 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:16.871833 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:16.872897 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:16.872908 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:16.872925 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:16.873618 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:16.873722 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:16.873749 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:16.873864 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:16.874011 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 06:56:16.874293 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":341,"threads_started":1}
I20260812 06:56:16.875872 20798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: Bootstrap starting.
I20260812 06:56:16.879086 20798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:16.879972 20798 log.cc:824] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:16.882586 20798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: No bootstrap required, opened a new log
I20260812 06:56:16.889438 20798 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } }
I20260812 06:56:16.889743 20798 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:16.889833 20798 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3af700a0e4f4fdc83f1bc851e01a62a, State: Initialized, Role: FOLLOWER
I20260812 06:56:16.890323 20798 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } }
I20260812 06:56:16.890571 20798 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:16.890699 20798 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:16.890843 20798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:16.892858 20798 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } }
I20260812 06:56:16.893362 20798 leader_election.cc:302] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3af700a0e4f4fdc83f1bc851e01a62a; no voters:
I20260812 06:56:16.893905 20798 leader_election.cc:288] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:16.894023 20806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:16.894701 20806 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [term 1 LEADER]: Becoming Leader. State: Replica: c3af700a0e4f4fdc83f1bc851e01a62a, State: Running, Role: LEADER
I20260812 06:56:16.895181 20806 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } }
I20260812 06:56:16.895707 20798 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:16.897743 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3af700a0e4f4fdc83f1bc851e01a62a. Latest consensus state: current_term: 1 leader_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } } }
I20260812 06:56:16.898173 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:16.898324 20808 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3af700a0e4f4fdc83f1bc851e01a62a" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46367 } } }
I20260812 06:56:16.898672 20808 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:16.900013 20822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:16.903833 20822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:16.911298 20822 catalog_manager.cc:1347] Generated new cluster ID: fd91e6ec6de84ca29ca9a69cae77303e
I20260812 06:56:16.911386 20822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:16.923331 20822 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:16.924394 20822 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:16.929940 20822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: Generated new TSK 0
I20260812 06:56:16.930660 20822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:16.945420 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.65:0
--local_ip_for_outbound_sockets=127.17.69.65
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--builtin_ntp_servers=127.17.69.84:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 06:56:17.269742 20847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:17.269982 20847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:17.270102 20847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:17.276993 20847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:17.277071 20847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:17.277206 20847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.65
I20260812 06:56:17.285868 20847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.69.65
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.65
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:17.286953 20847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:17.288528 20847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:17.300516 20865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.300577 20862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.300763 20861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:17.301399 20847 server_base.cc:1034] running on GCE node
I20260812 06:56:17.301946 20847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:17.302824 20847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:17.304028 20847 hybrid_clock.cc:648] HybridClock initialized: now 1786517777303979 us; error 42 us; skew 500 ppm
I20260812 06:56:17.304301 20847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:17.306313 20847 webserver.cc:460] Webserver started at http://127.17.69.65:38313/ using document root <none> and password file <none>
I20260812 06:56:17.306962 20847 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:17.307060 20847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:17.307339 20847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:17.309916 20847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data/instance:
uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "a928c3c28334b8865c2ccc9b9f60da32"
server_key_iv: "8deb9c781f941cfcc123eb4a915ca8a2"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.310683 20847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance:
uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "a928c3c28334b8865c2ccc9b9f60da32"
server_key_iv: "8deb9c781f941cfcc123eb4a915ca8a2"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.315382 20847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:17.318516 20880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.319654 20847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:17.319788 20847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal
uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "a928c3c28334b8865c2ccc9b9f60da32"
server_key_iv: "8deb9c781f941cfcc123eb4a915ca8a2"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.319967 20847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:17.335371 20847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:17.336025 20847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:17.336280 20847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:17.337105 20847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:17.338378 20847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:17.338474 20847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.338572 20847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:17.338651 20847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.367784 20847 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.65:44117
I20260812 06:56:17.367842 21083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.65:44117 every 8 connection(s)
I20260812 06:56:17.368948 20847 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/data/info.pb
I20260812 06:56:17.375427 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 20847
I20260812 06:56:17.375607 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal/instance
I20260812 06:56:17.376005 17685 external_mini_cluster.cc:1428] Setting key 8302e9e8a91e92ac7606e6b1b54af018
I20260812 06:56:17.378461 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.66:0
--local_ip_for_outbound_sockets=127.17.69.66
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--builtin_ntp_servers=127.17.69.84:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:17.380700 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.371263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.65:55943 (local address 127.17.69.126:46367)
0812 06:56:17.371436 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.371441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.372651 (+ 1210us) server_negotiation.cc:407] Connection header received
0812 06:56:17.374305 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.374310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.374409 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.374495 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:17.375684 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.376446 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.378102 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.378562 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.378679 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.380005 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:17.380011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:17.380015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:17.380035 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:17.380075 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:17.380092 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:17.380181 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.380338 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.380434 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:56:17.381510 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.371670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:46367 (local address 127.17.69.65:55943)
0812 06:56:17.372493 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.372534 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.374066 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.374734 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.374741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.374820 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:17.375510 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.375536 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.376588 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.376591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.377965 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.377978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.378888 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.378893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.379081 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.379704 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:17.379706 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:17.379741 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:17.379807 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:17.380175 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:17.380181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:17.380186 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.380307 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.380563 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":254,"threads_started":1}
I20260812 06:56:17.382792 21087 heartbeater.cc:347] Connected to a master server at 127.17.69.126:46367
I20260812 06:56:17.383067 21087 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:17.383868 21087 heartbeater.cc:511] Master 127.17.69.126:46367 requested a full tablet report, sending...
I20260812 06:56:17.385738 20727 ts_manager.cc:194] Registered new tserver with Master: 69fc7767f2ca42f9a0c02fe9be2f5d2b (127.17.69.65:44117)
I20260812 06:56:17.387156 20727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.65:55943
W20260812 06:56:17.697305 21092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:17.697530 21092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:17.697615 21092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:17.704522 21092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:17.704602 21092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:17.704708 21092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.66
I20260812 06:56:17.713083 21092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.69.66
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.66
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:17.714077 21092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:17.715613 21092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:17.727330 21101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.727300 21099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:17.727631 21098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:17.728217 21092 server_base.cc:1034] running on GCE node
I20260812 06:56:17.728757 21092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:17.729578 21092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:17.730773 21092 hybrid_clock.cc:648] HybridClock initialized: now 1786517777730724 us; error 41 us; skew 500 ppm
I20260812 06:56:17.731052 21092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:17.733014 21092 webserver.cc:460] Webserver started at http://127.17.69.66:41353/ using document root <none> and password file <none>
I20260812 06:56:17.733628 21092 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:17.733731 21092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:17.734007 21092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:17.736636 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data/instance:
uuid: "1e975f57cab84e37a50439315e18ce1d"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "ea0638c4438152dc7f0be4a583ef9dda"
server_key_iv: "de9d3038d3a2ea64e7c247e5ae30e6c1"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.737367 21092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance:
uuid: "1e975f57cab84e37a50439315e18ce1d"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "ea0638c4438152dc7f0be4a583ef9dda"
server_key_iv: "de9d3038d3a2ea64e7c247e5ae30e6c1"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.742033 21092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 06:56:17.745129 21112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.746222 21092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:56:17.746397 21092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal
uuid: "1e975f57cab84e37a50439315e18ce1d"
format_stamp: "Formatted at 2026-08-12 06:56:17 on dist-test-slave-6zbq"
server_key: "ea0638c4438152dc7f0be4a583ef9dda"
server_key_iv: "de9d3038d3a2ea64e7c247e5ae30e6c1"
server_key_version: "encryptionkey@0"
I20260812 06:56:17.746582 21092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:17.759119 21092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:17.759766 21092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:17.760017 21092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:17.760874 21092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:17.762164 21092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:17.762259 21092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.762377 21092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:17.762450 21092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:17.790800 21092 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.66:41585
I20260812 06:56:17.790859 21295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.66:41585 every 8 connection(s)
I20260812 06:56:17.791965 21092 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/data/info.pb
I20260812 06:56:17.798667 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21092
I20260812 06:56:17.798853 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal/instance
I20260812 06:56:17.799420 17685 external_mini_cluster.cc:1428] Setting key c02c12ee69ab78f65521ce8fa9c5b7f0
I20260812 06:56:17.802256 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.69.67:0
--local_ip_for_outbound_sockets=127.17.69.67
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--builtin_ntp_servers=127.17.69.84:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 06:56:17.804355 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.794291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.66:35455 (local address 127.17.69.126:46367)
0812 06:56:17.794482 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:17.794487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:17.795767 (+ 1280us) server_negotiation.cc:407] Connection header received
0812 06:56:17.797513 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:17.797518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:17.797590 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:17.797691 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:17.798765 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.799869 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.801644 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.802240 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.802406 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.803645 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:17.803652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:17.803656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:17.803678 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:17.803718 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:17.803732 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:17.803821 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:17.803984 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 06:56:17.804082 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 06:56:17.805146 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:17.794750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:46367 (local address 127.17.69.66:35455)
0812 06:56:17.795585 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:17.795657 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:17.797274 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:17.797853 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:17.797861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:17.797939 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:17.798543 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.798563 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.800029 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.800034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.801495 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:17.801508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:17.802475 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:17.802483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:17.802704 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:17.803335 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:17.803337 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:17.803380 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:17.803446 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:17.803815 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:17.803820 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:17.803825 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 06:56:17.803945 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 06:56:17.804201 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":252,"threads_started":1}
I20260812 06:56:17.806386 21296 heartbeater.cc:347] Connected to a master server at 127.17.69.126:46367
I20260812 06:56:17.806681 21296 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:17.807474 21296 heartbeater.cc:511] Master 127.17.69.126:46367 requested a full tablet report, sending...
I20260812 06:56:17.808822 20727 ts_manager.cc:194] Registered new tserver with Master: 1e975f57cab84e37a50439315e18ce1d (127.17.69.66:41585)
I20260812 06:56:17.809587 20727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.66:35455
W20260812 06:56:18.177035 21300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:18.177300 21300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:18.177429 21300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:18.184332 21300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:18.184423 21300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:18.184563 21300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.69.67
I20260812 06:56:18.193310 21300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.69.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.69.67
--webserver_port=0
--tserver_master_addrs=127.17.69.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.69.67
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:18.194491 21300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:18.196233 21300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:18.208595 21309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:18.208544 21316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:18.208632 21312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:18.209635 21300 server_base.cc:1034] running on GCE node
I20260812 06:56:18.210168 21300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:18.211071 21300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:18.212266 21300 hybrid_clock.cc:648] HybridClock initialized: now 1786517778212222 us; error 33 us; skew 500 ppm
I20260812 06:56:18.212534 21300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 06:56:18.214602 21300 webserver.cc:460] Webserver started at http://127.17.69.67:46441/ using document root <none> and password file <none>
I20260812 06:56:18.215210 21300 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:18.215310 21300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:18.215587 21300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:18.218357 21300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data/instance:
uuid: "8e4d63952e9342309e3e00e6b5e56440"
format_stamp: "Formatted at 2026-08-12 06:56:18 on dist-test-slave-6zbq"
server_key: "7f5226fa9226b3e700317aa48761c589"
server_key_iv: "b9be01fe5553862a993678901c6f79a6"
server_key_version: "encryptionkey@0"
I20260812 06:56:18.219113 21300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance:
uuid: "8e4d63952e9342309e3e00e6b5e56440"
format_stamp: "Formatted at 2026-08-12 06:56:18 on dist-test-slave-6zbq"
server_key: "7f5226fa9226b3e700317aa48761c589"
server_key_iv: "b9be01fe5553862a993678901c6f79a6"
server_key_version: "encryptionkey@0"
I20260812 06:56:18.223852 21300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:18.226989 21327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:18.228149 21300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:18.228293 21300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal
uuid: "8e4d63952e9342309e3e00e6b5e56440"
format_stamp: "Formatted at 2026-08-12 06:56:18 on dist-test-slave-6zbq"
server_key: "7f5226fa9226b3e700317aa48761c589"
server_key_iv: "b9be01fe5553862a993678901c6f79a6"
server_key_version: "encryptionkey@0"
I20260812 06:56:18.228488 21300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:18.242089 21300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:18.242818 21300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:18.243076 21300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:18.243920 21300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 06:56:18.245281 21300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:56:18.245375 21300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:18.245474 21300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 06:56:18.245541 21300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:18.274744 21300 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.67:42189
I20260812 06:56:18.274839 21515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.67:42189 every 8 connection(s)
I20260812 06:56:18.276149 21300 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/data/info.pb
I20260812 06:56:18.286417 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21300
I20260812 06:56:18.286597 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal/instance
I20260812 06:56:18.287101 17685 external_mini_cluster.cc:1428] Setting key 55780cd0b80c99cd2a1b508ead4befa3
I20260812 06:56:18.287367 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.278484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:33907 (local address 127.17.69.126:46367)
0812 06:56:18.278687 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.278694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.279963 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 06:56:18.281582 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.281586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.281654 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.281744 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.282703 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.283407 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.284855 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.285258 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.285353 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.286579 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.286586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.286591 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:18.286613 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.286654 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:18.286672 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.286780 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.286962 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.287061 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:56:18.288260 21518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.278971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:46367 (local address 127.17.69.67:33907)
0812 06:56:18.279806 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.279839 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.281364 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.281908 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.281915 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.281991 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:18.282545 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.282563 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.283545 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.283548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.284726 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.284738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.285435 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.285440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.285639 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.286244 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 06:56:18.286247 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:18.286282 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 06:56:18.286375 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:18.286774 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:18.286782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:18.286788 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.286913 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.287164 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":264,"threads_started":1}
I20260812 06:56:18.289698 21516 heartbeater.cc:347] Connected to a master server at 127.17.69.126:46367
I20260812 06:56:18.289985 21516 heartbeater.cc:464] Registering TS with master...
I20260812 06:56:18.290699 21516 heartbeater.cc:511] Master 127.17.69.126:46367 requested a full tablet report, sending...
I20260812 06:56:18.291903 20727 ts_manager.cc:194] Registered new tserver with Master: 8e4d63952e9342309e3e00e6b5e56440 (127.17.69.67:42189)
I20260812 06:56:18.292639 20727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.69.67:33907
I20260812 06:56:18.302019 17685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 06:56:18.311548 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.305914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.17.69.126:46367)
0812 06:56:18.306086 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.306092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.306514 (+ 422us) server_negotiation.cc:407] Connection header received
0812 06:56:18.306717 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.306721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.306792 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.306872 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.307345 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.308099 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.309420 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.309829 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.309935 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.310865 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.310871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.310876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:18.310893 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.310925 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:18.310936 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.311010 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.311149 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.311241 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 06:56:18.314941 20727 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 06:56:18.321578 20727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:56:18.348894 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.333367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.17.69.66:41585)
0812 06:56:18.338399 (+ 5032us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.338409 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.338806 (+ 397us) server_negotiation.cc:407] Connection header received
0812 06:56:18.339132 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.339139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.339377 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.339542 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.340636 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.342322 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.345847 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.347375 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.347482 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.347715 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.348382 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.348547 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4839,"thread_start_us":4722,"threads_started":1}
I20260812 06:56:18.349834 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.332846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41585 (local address 127.0.0.1:54416)
0812 06:56:18.338703 (+ 5857us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.338719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.338917 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.339830 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.339839 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.339865 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.340474 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.340484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.343021 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.343027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.345613 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.345629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.347862 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.347868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.348113 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.348171 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.349190 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.349373 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5744,"mutex_wait_us":6,"thread_start_us":705,"threads_started":1}
I20260812 06:56:18.349990 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.332945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.17.69.67:42189)
0812 06:56:18.338857 (+ 5912us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.338862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.338891 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:56:18.338989 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.338996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.339250 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.340090 (+ 840us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.340372 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.343437 (+ 3065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.347582 (+ 4145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.348746 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.348867 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.349123 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.349421 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.349580 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5769,"thread_start_us":5635,"threads_started":1}
I20260812 06:56:18.349856 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.332680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.67:42189 (local address 127.0.0.1:55968)
0812 06:56:18.333537 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.333586 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.333756 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.339502 (+ 5746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.339512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.339545 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.340059 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.340073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.343607 (+ 3534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.343618 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.346025 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.346041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.348923 (+ 2882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.348930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.349216 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.349247 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.349331 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.349437 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":22,"thread_start_us":5769,"threads_started":1}
I20260812 06:56:18.353422 21406 tablet_service.cc:1467] Processing CreateTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba (DEFAULT_TABLE table=test-table [id=a251fad7917a4da898a60c4df36d6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:18.353665 21193 tablet_service.cc:1467] Processing CreateTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba (DEFAULT_TABLE table=test-table [id=a251fad7917a4da898a60c4df36d6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:18.354912 21193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fac79d8ed5d4344b1cdfeba382e26ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:18.354951 21406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fac79d8ed5d4344b1cdfeba382e26ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:18.355996 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.339028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44117 (local address 127.0.0.1:53934)
0812 06:56:18.347837 (+ 8809us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.347855 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.347960 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.349290 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.349295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.349319 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.349702 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.349718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.351478 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.351483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.353376 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.353390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.355307 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.355313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.355475 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.355508 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.355596 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.355723 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8690,"thread_start_us":10088,"threads_started":1}
I20260812 06:56:18.356168 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.339096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.17.69.65:44117)
0812 06:56:18.348407 (+ 9311us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.348415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.348442 (+ 27us) server_negotiation.cc:407] Connection header received
0812 06:56:18.348529 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.348536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.348764 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.348943 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.350022 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.351314 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.353575 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.355099 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.355232 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.355456 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.355674 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.355829 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9148,"thread_start_us":9051,"threads_started":1}
I20260812 06:56:18.359642 20963 tablet_service.cc:1467] Processing CreateTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba (DEFAULT_TABLE table=test-table [id=a251fad7917a4da898a60c4df36d6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:56:18.360968 20963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fac79d8ed5d4344b1cdfeba382e26ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:18.365792 21543 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Bootstrap starting.
I20260812 06:56:18.368523 21545 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Bootstrap starting.
I20260812 06:56:18.371840 21545 tablet_bootstrap.cc:654] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:18.373126 21545 log.cc:824] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:18.373732 21543 tablet_bootstrap.cc:654] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:18.374972 21543 log.cc:824] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:18.378677 21547 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Bootstrap starting.
I20260812 06:56:18.380359 21545 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: No bootstrap required, opened a new log
I20260812 06:56:18.380625 21545 ts_tablet_manager.cc:1397] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260812 06:56:18.382032 21547 tablet_bootstrap.cc:654] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:18.382480 21543 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: No bootstrap required, opened a new log
I20260812 06:56:18.382699 21543 ts_tablet_manager.cc:1397] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.005s
I20260812 06:56:18.383148 21547 log.cc:824] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:18.385905 21547 tablet_bootstrap.cc:492] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: No bootstrap required, opened a new log
I20260812 06:56:18.386137 21547 ts_tablet_manager.cc:1397] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 06:56:18.389724 21543 raft_consensus.cc:348] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.390070 21543 raft_consensus.cc:374] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:18.390138 21087 heartbeater.cc:503] Master 127.17.69.126:46367 was elected leader, sending a full tablet report...
I20260812 06:56:18.390175 21543 raft_consensus.cc:729] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4d63952e9342309e3e00e6b5e56440, State: Initialized, Role: FOLLOWER
I20260812 06:56:18.390250 21545 raft_consensus.cc:348] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.390663 21545 raft_consensus.cc:374] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:18.390753 21545 raft_consensus.cc:729] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e975f57cab84e37a50439315e18ce1d, State: Initialized, Role: FOLLOWER
I20260812 06:56:18.390885 21543 consensus_queue.cc:260] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.391387 21545 consensus_queue.cc:260] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.395490 21516 heartbeater.cc:503] Master 127.17.69.126:46367 was elected leader, sending a full tablet report...
I20260812 06:56:18.396184 21543 ts_tablet_manager.cc:1428] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 06:56:18.396420 21547 raft_consensus.cc:348] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.396855 21547 raft_consensus.cc:374] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:18.396951 21547 raft_consensus.cc:729] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fc7767f2ca42f9a0c02fe9be2f5d2b, State: Initialized, Role: FOLLOWER
I20260812 06:56:18.397332 21296 heartbeater.cc:503] Master 127.17.69.126:46367 was elected leader, sending a full tablet report...
I20260812 06:56:18.397567 21547 consensus_queue.cc:260] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.398814 21545 ts_tablet_manager.cc:1428] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Time spent starting tablet: real 0.018s user 0.013s sys 0.006s
I20260812 06:56:18.401237 21547 ts_tablet_manager.cc:1428] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
W20260812 06:56:18.527676 21517 tablet.cc:2369] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:56:18.543921 21297 tablet.cc:2369] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:18.582464 21552 raft_consensus.cc:482] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:56:18.582734 21552 raft_consensus.cc:504] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.584359 21552 leader_election.cc:288] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e975f57cab84e37a50439315e18ce1d (127.17.69.66:41585), 69fc7767f2ca42f9a0c02fe9be2f5d2b (127.17.69.65:44117)
I20260812 06:56:18.590570 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.584819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:58117 (local address 127.17.69.65:44117)
0812 06:56:18.585000 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.585005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.585063 (+ 58us) server_negotiation.cc:407] Connection header received
0812 06:56:18.585268 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.585273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.585367 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.585469 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.585896 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.586856 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.588757 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.589804 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.589918 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.589966 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.590255 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.590370 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:56:18.590888 21518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.584704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.65:44117 (local address 127.17.69.67:58117)
0812 06:56:18.584971 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.584989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.585131 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.585528 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.585533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.585559 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.585729 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.585739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.586972 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.586978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.588558 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.588570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.589938 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.589944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.590111 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.590145 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.590238 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.590393 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260812 06:56:18.591346 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.584716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.69.67:51917 (local address 127.17.69.66:41585)
0812 06:56:18.584892 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.584898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.585789 (+ 891us) server_negotiation.cc:407] Connection header received
0812 06:56:18.585931 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.585935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.586003 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.586069 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 06:56:18.586698 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.587619 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.589560 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.590556 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.590711 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.590767 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.591042 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.591169 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 06:56:18.591436 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.584717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.66:41585 (local address 127.17.69.67:51917)
0812 06:56:18.585689 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:18.585703 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:18.585811 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:18.586136 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:18.586141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:18.586175 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 06:56:18.586543 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.586556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.587788 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.587792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.589380 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:18.589395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.590718 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.590725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:18.590878 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.590902 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 06:56:18.590989 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 06:56:18.591098 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":199,"thread_start_us":413,"threads_started":1}
I20260812 06:56:18.591902 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fac79d8ed5d4344b1cdfeba382e26ba" candidate_uuid: "8e4d63952e9342309e3e00e6b5e56440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" is_pre_election: true
I20260812 06:56:18.592396 20995 raft_consensus.cc:2454] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e4d63952e9342309e3e00e6b5e56440 in term 0.
I20260812 06:56:18.592352 21224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fac79d8ed5d4344b1cdfeba382e26ba" candidate_uuid: "8e4d63952e9342309e3e00e6b5e56440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e975f57cab84e37a50439315e18ce1d" is_pre_election: true
I20260812 06:56:18.592833 21224 raft_consensus.cc:2454] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e4d63952e9342309e3e00e6b5e56440 in term 0.
I20260812 06:56:18.592972 21334 leader_election.cc:302] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69fc7767f2ca42f9a0c02fe9be2f5d2b, 8e4d63952e9342309e3e00e6b5e56440; no voters:
I20260812 06:56:18.593338 21552 raft_consensus.cc:2781] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:56:18.593487 21552 raft_consensus.cc:482] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:56:18.593593 21552 raft_consensus.cc:3037] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.595714 21552 raft_consensus.cc:504] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.596276 21552 leader_election.cc:288] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [CANDIDATE]: Term 1 election: Requested vote from peers 1e975f57cab84e37a50439315e18ce1d (127.17.69.66:41585), 69fc7767f2ca42f9a0c02fe9be2f5d2b (127.17.69.65:44117)
I20260812 06:56:18.596632 21224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fac79d8ed5d4344b1cdfeba382e26ba" candidate_uuid: "8e4d63952e9342309e3e00e6b5e56440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e975f57cab84e37a50439315e18ce1d"
I20260812 06:56:18.596763 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fac79d8ed5d4344b1cdfeba382e26ba" candidate_uuid: "8e4d63952e9342309e3e00e6b5e56440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b"
I20260812 06:56:18.596830 21224 raft_consensus.cc:3037] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.596976 20995 raft_consensus.cc:3037] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:18.599974 21224 raft_consensus.cc:2454] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e4d63952e9342309e3e00e6b5e56440 in term 1.
I20260812 06:56:18.600003 20995 raft_consensus.cc:2454] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e4d63952e9342309e3e00e6b5e56440 in term 1.
I20260812 06:56:18.600380 21337 leader_election.cc:302] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e975f57cab84e37a50439315e18ce1d, 8e4d63952e9342309e3e00e6b5e56440; no voters:
I20260812 06:56:18.600629 21552 raft_consensus.cc:2781] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:18.601037 21552 raft_consensus.cc:686] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 1 LEADER]: Becoming Leader. State: Replica: 8e4d63952e9342309e3e00e6b5e56440, State: Running, Role: LEADER
I20260812 06:56:18.601673 21552 consensus_queue.cc:237] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } }
I20260812 06:56:18.606768 20727 catalog_manager.cc:5495] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e4d63952e9342309e3e00e6b5e56440 (127.17.69.67). New cstate: current_term: 1 leader_uuid: "8e4d63952e9342309e3e00e6b5e56440" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e4d63952e9342309e3e00e6b5e56440" member_type: VOTER last_known_addr { host: "127.17.69.67" port: 42189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 } health_report { overall_health: UNKNOWN } } }
W20260812 06:56:18.621582 21088 tablet.cc:2369] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:56:18.637941 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.633132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.17.69.67:42189)
0812 06:56:18.633285 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.633289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.633435 (+ 146us) server_negotiation.cc:407] Connection header received
0812 06:56:18.633600 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.633604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.633675 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.633790 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:56:18.634059 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.634822 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.636163 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.636561 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.636649 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.637030 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:56:18.637516 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:56:18.637625 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.637707 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.637767 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 06:56:18.645342 20995 raft_consensus.cc:1261] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 1 FOLLOWER]: Refusing update from remote peer 8e4d63952e9342309e3e00e6b5e56440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:18.645344 21224 raft_consensus.cc:1261] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 1 FOLLOWER]: Refusing update from remote peer 8e4d63952e9342309e3e00e6b5e56440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:56:18.646191 21552 consensus_queue.cc:1035] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e975f57cab84e37a50439315e18ce1d" member_type: VOTER last_known_addr { host: "127.17.69.66" port: 41585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:18.646497 21558 consensus_queue.cc:1035] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69fc7767f2ca42f9a0c02fe9be2f5d2b" member_type: VOTER last_known_addr { host: "127.17.69.65" port: 44117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:56:18.658728 21568 mvcc.cc:205] Tried to move back new op lower bound from 7317576821322440704 to 7317576821156126720. Current Snapshot: MvccSnapshot[applied={T|T < 7317576821322440704}]
I20260812 06:56:18.668952 20727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:18.669176 20727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 06:56:18.672489 20727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3af700a0e4f4fdc83f1bc851e01a62a: Sending DeleteTablet for 3 replicas of tablet 2fac79d8ed5d4344b1cdfeba382e26ba
I20260812 06:56:18.673599 21406 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='slave'} at 127.0.0.1:55968
I20260812 06:56:18.673584 21193 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='slave'} at 127.0.0.1:54416
I20260812 06:56:18.673784 20963 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fac79d8ed5d4344b1cdfeba382e26ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 06:56:18 UTC) from {username='slave'} at 127.0.0.1:53934
I20260812 06:56:18.675588 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20847
I20260812 06:56:18.683666 21582 tablet_replica.cc:330] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: stopping tablet replica
I20260812 06:56:18.684175 21582 raft_consensus.cc:2229] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:18.684747 21582 raft_consensus.cc:2258] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.686949 21582 ts_tablet_manager.cc:1905] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.687490 21580 tablet_replica.cc:330] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: stopping tablet replica
I20260812 06:56:18.687914 21580 raft_consensus.cc:2229] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:56:18.694725 21580 raft_consensus.cc:2258] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.695091 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.678939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.17.69.65:44117)
0812 06:56:18.679131 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.679138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.682815 (+ 3677us) server_negotiation.cc:407] Connection header received
0812 06:56:18.683029 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.683035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.683168 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.683278 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.687864 (+ 4586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.689102 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.691100 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.691805 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.691904 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.693377 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.693388 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.693405 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:18.694159 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.694225 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:18.694250 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.694378 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.694611 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.694729 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:18.696197 21076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.697180 21580 ts_tablet_manager.cc:1905] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.703785 21580 ts_tablet_manager.cc:1918] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.704300 21581 tablet_replica.cc:330] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: stopping tablet replica
I20260812 06:56:18.704674 21581 raft_consensus.cc:2229] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:56:18.705291 21581 raft_consensus.cc:2258] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:56:18.707978 21581 ts_tablet_manager.cc:1905] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 06:56:18.706202 21580 log.cc:1196] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-1/wal/wals/2fac79d8ed5d4344b1cdfeba382e26ba
I20260812 06:56:18.709142 21580 ts_tablet_manager.cc:1939] T 2fac79d8ed5d4344b1cdfeba382e26ba P 1e975f57cab84e37a50439315e18ce1d: Deleting consensus metadata
I20260812 06:56:18.710647 20701 catalog_manager.cc:4841] TS 1e975f57cab84e37a50439315e18ce1d (127.17.69.66:41585): tablet 2fac79d8ed5d4344b1cdfeba382e26ba (table test-table [id=a251fad7917a4da898a60c4df36d6d13]) successfully deleted
I20260812 06:56:18.716929 21581 ts_tablet_manager.cc:1918] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.717085 21581 log.cc:1196] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-2/wal/wals/2fac79d8ed5d4344b1cdfeba382e26ba
I20260812 06:56:18.717624 21581 ts_tablet_manager.cc:1939] T 2fac79d8ed5d4344b1cdfeba382e26ba P 8e4d63952e9342309e3e00e6b5e56440: Deleting consensus metadata
I20260812 06:56:18.719233 20697 catalog_manager.cc:4841] TS 8e4d63952e9342309e3e00e6b5e56440 (127.17.69.67:42189): tablet 2fac79d8ed5d4344b1cdfeba382e26ba (table test-table [id=a251fad7917a4da898a60c4df36d6d13]) successfully deleted
I20260812 06:56:18.791586 21582 ts_tablet_manager.cc:1918] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 06:56:18.791851 21582 log.cc:1196] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Deleting WAL directory at /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786517466184829-17685-0/minicluster-data/ts-0/wal/wals/2fac79d8ed5d4344b1cdfeba382e26ba
I20260812 06:56:18.792554 21582 ts_tablet_manager.cc:1939] T 2fac79d8ed5d4344b1cdfeba382e26ba P 69fc7767f2ca42f9a0c02fe9be2f5d2b: Deleting consensus metadata
I20260812 06:56:18.794566 20697 catalog_manager.cc:4841] TS 69fc7767f2ca42f9a0c02fe9be2f5d2b (127.17.69.65:44117): tablet 2fac79d8ed5d4344b1cdfeba382e26ba (table test-table [id=a251fad7917a4da898a60c4df36d6d13]) successfully deleted
I20260812 06:56:18.794858 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20847
I20260812 06:56:18.817344 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21092
I20260812 06:56:18.824209 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.818109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54424 (local address 127.17.69.66:41585)
0812 06:56:18.818317 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.818323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.818525 (+ 202us) server_negotiation.cc:407] Connection header received
0812 06:56:18.818715 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.818719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.818795 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.818890 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.819277 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.820074 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.821292 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.821710 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.821816 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.822856 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.822865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.822877 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:18.823435 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.823483 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:18.823503 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.823610 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.823752 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.823859 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 06:56:18.824848 21288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:18.925087 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21092
I20260812 06:56:18.947876 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21300
I20260812 06:56:18.957564 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:18.948619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.17.69.67:42189)
0812 06:56:18.948828 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:18.948834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:18.948902 (+ 68us) server_negotiation.cc:407] Connection header received
0812 06:56:18.949093 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:18.949098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:18.949208 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:18.949308 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:18.949702 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.950546 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.951795 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:18.952236 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:18.952343 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:18.956193 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:18.956202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:18.956217 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:18.956816 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:18.956866 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:18.956895 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:18.956986 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:18.957126 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 06:56:18.957243 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 06:56:18.958244 21510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:19.076462 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21300
I20260812 06:56:19.103775 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 20670
I20260812 06:56:19.104856 20790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:19.209612 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 20670
2026-08-12T06:56:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2864 ms)
[----------] 6 tests from EncryptionPolicyTest (21168 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:19 dist-test-slave-6zbq krb5kdc[21591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:19 dist-test-slave-6zbq krb5kdc[21591](info): set up 2 sockets
Aug 12 06:56:19 dist-test-slave-6zbq krb5kdc[21591](info): commencing operation
krb5kdc: starting...
W20260812 06:56:20.545536 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:20 dist-test-slave-6zbq krb5kdc[21591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:20.812103 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:21.134646 21610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:21.134891 21610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:21.135022 21610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:21.141885 21610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:21.141986 21610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:21.142057 21610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:21.142143 21610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:21.142195 21610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:21.152066 21610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:21.153158 21610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:21.154397 21610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:21.165275 21623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:21.165307 21626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:21.165235 21622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:21.166194 21610 server_base.cc:1034] running on GCE node
I20260812 06:56:21.166929 21610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:21.168177 21610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:21.169380 21610 hybrid_clock.cc:648] HybridClock initialized: now 1786517781169301 us; error 68 us; skew 500 ppm
I20260812 06:56:21.169696 21610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:21 dist-test-slave-6zbq krb5kdc[21591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:21.172631 21610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:21.173988 21610 webserver.cc:460] Webserver started at http://127.0.0.1:40335/ using document root <none> and password file <none>
I20260812 06:56:21.174633 21610 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:21.174752 21610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:21.175024 21610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:21.177745 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "214ceb79061c49178ebff0e2e92ca6a5"
format_stamp: "Formatted at 2026-08-12 06:56:21 on dist-test-slave-6zbq"
server_key: "7321cc15b53e0da8b10964a910d16679"
server_key_iv: "480a31185eb810f072bc612e4f920751"
server_key_version: "encryptionkey@0"
I20260812 06:56:21.178520 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "214ceb79061c49178ebff0e2e92ca6a5"
format_stamp: "Formatted at 2026-08-12 06:56:21 on dist-test-slave-6zbq"
server_key: "7321cc15b53e0da8b10964a910d16679"
server_key_iv: "480a31185eb810f072bc612e4f920751"
server_key_version: "encryptionkey@0"
I20260812 06:56:21.183116 21610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 06:56:21.186206 21647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:21.187377 21610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:21.187527 21610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "214ceb79061c49178ebff0e2e92ca6a5"
format_stamp: "Formatted at 2026-08-12 06:56:21 on dist-test-slave-6zbq"
server_key: "7321cc15b53e0da8b10964a910d16679"
server_key_iv: "480a31185eb810f072bc612e4f920751"
server_key_version: "encryptionkey@0"
I20260812 06:56:21.187700 21610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:21.213398 21610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:21.217384 21610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:21.217641 21610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:21.234490 21610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38201
I20260812 06:56:21.234505 21736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38201 every 8 connection(s)
I20260812 06:56:21.235705 21610 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:21.237221 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21610
I20260812 06:56:21.237442 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:21.238296 17685 external_mini_cluster.cc:1428] Setting key 590be63f9f1427829b234e833afb4c53
I20260812 06:56:21.239815 21737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:21.250504 21737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5: Bootstrap starting.
Aug 12 06:56:21 dist-test-slave-6zbq krb5kdc[21591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:56:21.253473 21737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:21.254482 21737 log.cc:824] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:21.256939 21737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5: No bootstrap required, opened a new log
I20260812 06:56:21.259148 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.240143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40649 (local address 127.0.0.1:38201)
0812 06:56:21.240875 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.240885 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.240923 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:56:21.242533 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.242555 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.242914 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.243278 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.244653 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:21.245956 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:21.247355 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:21.248013 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:21.248201 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:21.252260 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.252267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.252280 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:21.252313 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.255232 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:21.255994 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:21.256000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:21.256006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:21.256163 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:21.256602 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:21.256606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:21.256608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:21.257119 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.257433 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.257903 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.258184 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":318,"threads_started":1}
I20260812 06:56:21.262779 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:21.264515 21737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260812 06:56:21.264822 21737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:21.264920 21737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 214ceb79061c49178ebff0e2e92ca6a5, State: Initialized, Role: FOLLOWER
I20260812 06:56:21.265484 21737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260812 06:56:21.265703 21737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:21.265843 21737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:21.266005 21737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:21.268690 21737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260812 06:56:21.269115 21737 leader_election.cc:302] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 214ceb79061c49178ebff0e2e92ca6a5; no voters:
I20260812 06:56:21.269641 21737 leader_election.cc:288] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:21.269796 21748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:21.270489 21748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [term 1 LEADER]: Becoming Leader. State: Replica: 214ceb79061c49178ebff0e2e92ca6a5, State: Running, Role: LEADER
I20260812 06:56:21.271000 21748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260812 06:56:21.272778 21748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 214ceb79061c49178ebff0e2e92ca6a5. Latest consensus state: current_term: 1 leader_uuid: "214ceb79061c49178ebff0e2e92ca6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260812 06:56:21.273043 21748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:21.274134 21751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "214ceb79061c49178ebff0e2e92ca6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214ceb79061c49178ebff0e2e92ca6a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260812 06:56:21.274804 21751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:21.275923 21737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:21.278743 21756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:21.283381 21756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:21.285178 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:21.267461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59457 (local address 127.0.0.1:38201)
0812 06:56:21.267595 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:21.267599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:21.268033 (+ 434us) server_negotiation.cc:407] Connection header received
0812 06:56:21.268148 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:21.268153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:21.268219 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:21.268305 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:21.269893 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:21.271071 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:21.272841 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:21.273522 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:21.273615 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:21.276872 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:21.276879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:21.276884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:21.276921 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:21.280625 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:21.281699 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:21.281705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:21.281709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:21.281896 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:21.282519 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:21.282524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:21.282528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:21.282900 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:21.284376 (+ 1476us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:21.284554 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 06:56:21.284739 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:56:21.296886 21756 catalog_manager.cc:1347] Generated new cluster ID: 85e052079f7d41b1a54a0f59c66aa7f8
I20260812 06:56:21.296983 21756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:21.307880 21756 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:21.309538 21756 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:21.321202 21756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 214ceb79061c49178ebff0e2e92ca6a5: Generated new TSK 0
I20260812 06:56:21.322069 21756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:21.337739 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21610
I20260812 06:56:21.339076 21731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:21.426774 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21610
W20260812 06:56:21.452196 21588 connection.cc:535] client connection to 127.0.0.1:38201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38201): Connection reset by peer (error 104)
2026-08-12T06:56:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:21 dist-test-slave-6zbq krb5kdc[21789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:21 dist-test-slave-6zbq krb5kdc[21789](info): set up 2 sockets
Aug 12 06:56:21 dist-test-slave-6zbq krb5kdc[21789](info): commencing operation
krb5kdc: starting...
W20260812 06:56:22.669034 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:22 dist-test-slave-6zbq krb5kdc[21789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:22.943595 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:23.276386 21811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:23.276646 21811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:23.276798 21811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:23.283607 21811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:23.283721 21811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:23.283799 21811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:23.283890 21811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:23.283951 21811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:23.293181 21811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:23.294363 21811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:23.295575 21811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:23.306870 21820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:23.306911 21827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:23.306939 21823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:23.307657 21811 server_base.cc:1034] running on GCE node
I20260812 06:56:23.308385 21811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:23.309715 21811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:23.310933 21811 hybrid_clock.cc:648] HybridClock initialized: now 1786517783310883 us; error 44 us; skew 500 ppm
I20260812 06:56:23.311262 21811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:23 dist-test-slave-6zbq krb5kdc[21789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:23.314211 21811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:23.315677 21811 webserver.cc:460] Webserver started at http://127.0.0.1:39019/ using document root <none> and password file <none>
I20260812 06:56:23.316296 21811 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:23.316421 21811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:23.316706 21811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:23.319581 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-6zbq"
server_key: "e48b41fe95d3d8e14cdfd4bf188d8335"
server_key_iv: "1108641ba12b40e15588acfa925e315c"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.320328 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-6zbq"
server_key: "e48b41fe95d3d8e14cdfd4bf188d8335"
server_key_iv: "1108641ba12b40e15588acfa925e315c"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.325139 21811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:23.328382 21837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:23.329576 21811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 06:56:23.329732 21811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c"
format_stamp: "Formatted at 2026-08-12 06:56:23 on dist-test-slave-6zbq"
server_key: "e48b41fe95d3d8e14cdfd4bf188d8335"
server_key_iv: "1108641ba12b40e15588acfa925e315c"
server_key_version: "encryptionkey@0"
I20260812 06:56:23.329900 21811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:23.348469 21811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:23.352124 21811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:23.352368 21811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:23.369244 21811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34829
I20260812 06:56:23.369300 21941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34829 every 8 connection(s)
I20260812 06:56:23.370524 21811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:23.373857 21942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:23.378433 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 21811
I20260812 06:56:23.378629 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:23.379194 17685 external_mini_cluster.cc:1428] Setting key cea16bd4bff9f2cb66f5fe9532a7a91f
I20260812 06:56:23.384495 21942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c: Bootstrap starting.
I20260812 06:56:23.387746 21942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:23.388849 21942 log.cc:824] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:23.391963 21942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c: No bootstrap required, opened a new log
I20260812 06:56:23.392002 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.380799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59799 (local address 127.0.0.1:34829)
0812 06:56:23.381531 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.381541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.381583 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:23.383310 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.383332 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.383682 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.383967 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:23.384559 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.385854 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.387641 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.388329 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.388460 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.389699 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:23.389711 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:23.389730 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:23.390459 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:23.390538 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:23.390563 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:23.390687 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:23.390949 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 06:56:23.391105 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":355,"threads_started":1}
I20260812 06:56:23.395485 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:23.400799 21942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260812 06:56:23.401225 21942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:23.401345 21942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6df6f7e7594f9f98a0e4c86c68bd0c, State: Initialized, Role: FOLLOWER
I20260812 06:56:23.401927 21942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260812 06:56:23.402148 21942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:23.402277 21942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:23.402452 21942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:23.405210 21942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260812 06:56:23.405740 21942 leader_election.cc:302] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e6df6f7e7594f9f98a0e4c86c68bd0c; no voters:
I20260812 06:56:23.406369 21942 leader_election.cc:288] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:23.406567 21954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:23.407030 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:23.400408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43137 (local address 127.0.0.1:34829)
0812 06:56:23.400538 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:23.400541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:23.400890 (+ 349us) server_negotiation.cc:407] Connection header received
0812 06:56:23.401061 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:23.401064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:23.401128 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:23.401223 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:23.401760 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.402659 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.404475 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:23.404997 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:23.405098 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:23.406169 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:23.406175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:23.406180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:23.406209 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:23.406282 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:23.406307 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:23.406457 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:23.406649 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 06:56:23.406700 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 06:56:23.407333 21954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [term 1 LEADER]: Becoming Leader. State: Replica: 3e6df6f7e7594f9f98a0e4c86c68bd0c, State: Running, Role: LEADER
I20260812 06:56:23.407887 21954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260812 06:56:23.409986 21954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e6df6f7e7594f9f98a0e4c86c68bd0c. Latest consensus state: current_term: 1 leader_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } } }
I20260812 06:56:23.410272 21954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.410761 21961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e6df6f7e7594f9f98a0e4c86c68bd0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } } }
I20260812 06:56:23.411096 21961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:23.411505 21942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:23.411722 21963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:23.422778 21963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:23.430516 21963 catalog_manager.cc:1347] Generated new cluster ID: 776d58e38b2e4f5891562b63e1743a83
I20260812 06:56:23.430629 21963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:23.442806 21963 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:23.444435 21963 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:23.451506 21963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e6df6f7e7594f9f98a0e4c86c68bd0c: Generated new TSK 0
I20260812 06:56:23.452147 21963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:23.466858 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21811
W20260812 06:56:23.467931 21934 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59799
W20260812 06:56:23.468607 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 21811: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:23.468780 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 21811
W20260812 06:56:23.492658 21785 connection.cc:535] client connection to 127.0.0.1:34829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34829): Connection reset by peer (error 104)
2026-08-12T06:56:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:23 dist-test-slave-6zbq krb5kdc[21993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:23 dist-test-slave-6zbq krb5kdc[21993](info): set up 2 sockets
Aug 12 06:56:23 dist-test-slave-6zbq krb5kdc[21993](info): commencing operation
krb5kdc: starting...
W20260812 06:56:24.748206 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:24 dist-test-slave-6zbq krb5kdc[21993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:25.026252 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:25.344758 22021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:25.345026 22021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:25.345124 22021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:25.351996 22021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:25.352088 22021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:25.352159 22021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:25.352221 22021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:25.352264 22021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:25.361469 22021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:25.362562 22021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:25.363704 22021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:25.374759 22030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:25.374766 22031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:25.374708 22035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:25.375650 22021 server_base.cc:1034] running on GCE node
I20260812 06:56:25.376338 22021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:25.377615 22021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:25.378813 22021 hybrid_clock.cc:648] HybridClock initialized: now 1786517785378773 us; error 34 us; skew 500 ppm
I20260812 06:56:25.379117 22021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:25 dist-test-slave-6zbq krb5kdc[21993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:25.385628 22021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:25.387060 22021 webserver.cc:460] Webserver started at http://127.0.0.1:40947/ using document root <none> and password file <none>
I20260812 06:56:25.387681 22021 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:25.387809 22021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:25.388091 22021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:25.390965 22021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "29be991f583e4ba4b034f99c540ed773"
format_stamp: "Formatted at 2026-08-12 06:56:25 on dist-test-slave-6zbq"
server_key: "ec9449bbe59c4f78a4ae12ad52f308d5"
server_key_iv: "8e7c654b300018cbbf4a1eceb6608a68"
server_key_version: "encryptionkey@0"
I20260812 06:56:25.391705 22021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "29be991f583e4ba4b034f99c540ed773"
format_stamp: "Formatted at 2026-08-12 06:56:25 on dist-test-slave-6zbq"
server_key: "ec9449bbe59c4f78a4ae12ad52f308d5"
server_key_iv: "8e7c654b300018cbbf4a1eceb6608a68"
server_key_version: "encryptionkey@0"
I20260812 06:56:25.396607 22021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 06:56:25.399850 22046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:25.401103 22021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:25.401258 22021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "29be991f583e4ba4b034f99c540ed773"
format_stamp: "Formatted at 2026-08-12 06:56:25 on dist-test-slave-6zbq"
server_key: "ec9449bbe59c4f78a4ae12ad52f308d5"
server_key_iv: "8e7c654b300018cbbf4a1eceb6608a68"
server_key_version: "encryptionkey@0"
I20260812 06:56:25.401437 22021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:25.406905 22021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:25.410640 22021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:25.410902 22021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:25.427516 22021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36101
I20260812 06:56:25.427569 22134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36101 every 8 connection(s)
I20260812 06:56:25.428733 22021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:25.430735 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22021
I20260812 06:56:25.430899 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:25.431311 17685 external_mini_cluster.cc:1428] Setting key c6be6391cfb665528e84388778d922ff
I20260812 06:56:25.432459 22135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:25.438398 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:25.432901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35735 (local address 127.0.0.1:36101)
0812 06:56:25.433657 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:25.433668 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:25.433707 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:25.435304 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:25.435324 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:25.435628 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:25.435939 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:25.436937 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:25.436947 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:25.436959 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:25.436973 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:25.437047 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:25.437066 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:25.437227 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:25.437329 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 06:56:25.437491 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":333,"threads_started":1}
I20260812 06:56:25.441645 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:25.444067 22135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773: Bootstrap starting.
I20260812 06:56:25.447024 22135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:25.448047 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:25.445732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43585 (local address 127.0.0.1:36101)
0812 06:56:25.445911 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:25.445917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:25.446321 (+ 404us) server_negotiation.cc:407] Connection header received
0812 06:56:25.446517 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:25.446522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:25.446567 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:25.446635 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:25.447431 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:25.447437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:25.447441 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:25.447459 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:25.447494 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:25.447508 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:25.447597 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:25.447711 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 06:56:25.447767 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:56:25.448208 22135 log.cc:824] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:25.450770 22135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773: No bootstrap required, opened a new log
I20260812 06:56:25.457562 22135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } }
I20260812 06:56:25.457903 22135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:25.458002 22135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29be991f583e4ba4b034f99c540ed773, State: Initialized, Role: FOLLOWER
I20260812 06:56:25.458575 22135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } }
I20260812 06:56:25.458786 22135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:25.458910 22135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:25.459051 22135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:25.460994 22135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } }
I20260812 06:56:25.461437 22135 leader_election.cc:302] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29be991f583e4ba4b034f99c540ed773; no voters:
I20260812 06:56:25.461934 22135 leader_election.cc:288] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:25.462049 22156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:25.462759 22156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [term 1 LEADER]: Becoming Leader. State: Replica: 29be991f583e4ba4b034f99c540ed773, State: Running, Role: LEADER
I20260812 06:56:25.463235 22156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } }
I20260812 06:56:25.463828 22135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:25.465683 22159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29be991f583e4ba4b034f99c540ed773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } } }
I20260812 06:56:25.466099 22159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:25.467662 22160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29be991f583e4ba4b034f99c540ed773. Latest consensus state: current_term: 1 leader_uuid: "29be991f583e4ba4b034f99c540ed773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29be991f583e4ba4b034f99c540ed773" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36101 } } }
I20260812 06:56:25.467950 22160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:25.471195 22171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:25.475102 22171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:25.482481 22171 catalog_manager.cc:1347] Generated new cluster ID: e562daf4a5b146bb9944367e7377c159
I20260812 06:56:25.482568 22171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:25.492818 22171 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:25.493165 22171 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:25.510546 22171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29be991f583e4ba4b034f99c540ed773: Generated new TSK 0
I20260812 06:56:25.511281 22171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:25.517746 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22021
W20260812 06:56:25.518894 22129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35735
W20260812 06:56:25.519474 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22021: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:25.519588 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22021
2026-08-12T06:56:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2054 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:25 dist-test-slave-6zbq krb5kdc[22191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:25 dist-test-slave-6zbq krb5kdc[22191](info): set up 2 sockets
Aug 12 06:56:25 dist-test-slave-6zbq krb5kdc[22191](info): commencing operation
krb5kdc: starting...
W20260812 06:56:26.796033 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:26 dist-test-slave-6zbq krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:27.072916 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:27.393882 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:27.394136 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:27.394235 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:27.401065 22215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:27.401177 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:27.401227 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:27.401291 22215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:27.401335 22215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:27.410437 22215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:27.411474 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:27.412611 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:27.423240 22222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:27.423209 22225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:27.423167 22228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:27.424155 22215 server_base.cc:1034] running on GCE node
I20260812 06:56:27.424809 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:27.426051 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:27.427253 22215 hybrid_clock.cc:648] HybridClock initialized: now 1786517787427216 us; error 25 us; skew 500 ppm
I20260812 06:56:27.427567 22215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:27 dist-test-slave-6zbq krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:27.430542 22215 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:27.431877 22215 webserver.cc:460] Webserver started at http://127.0.0.1:39255/ using document root <none> and password file <none>
I20260812 06:56:27.432466 22215 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:27.432580 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:27.432848 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:27.435586 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-6zbq"
server_key: "3391d04bf23aedb74328c79fa3ad6807"
server_key_iv: "f6a78f38bace893677ea3bab5243d94d"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.436307 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-6zbq"
server_key: "3391d04bf23aedb74328c79fa3ad6807"
server_key_iv: "f6a78f38bace893677ea3bab5243d94d"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.440892 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:27.444025 22252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:27.445144 22215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:56:27.445293 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c"
format_stamp: "Formatted at 2026-08-12 06:56:27 on dist-test-slave-6zbq"
server_key: "3391d04bf23aedb74328c79fa3ad6807"
server_key_iv: "f6a78f38bace893677ea3bab5243d94d"
server_key_version: "encryptionkey@0"
I20260812 06:56:27.445461 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:27.462857 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:27.466675 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:27.466913 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:27.483608 22215 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38665
I20260812 06:56:27.483636 22358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38665 every 8 connection(s)
I20260812 06:56:27.484783 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:27.485369 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22215
I20260812 06:56:27.485561 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:27.486078 17685 external_mini_cluster.cc:1428] Setting key 19bbfa61d810c79d6902edb58987422d
I20260812 06:56:27.488729 22359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:56:27 dist-test-slave-6zbq krb5kdc[22191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:56:27.498488 22359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c: Bootstrap starting.
I20260812 06:56:27.501497 22359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:27.502499 22359 log.cc:824] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:27.505517 22359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c: No bootstrap required, opened a new log
I20260812 06:56:27.506390 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:27.487672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56915 (local address 127.0.0.1:38665)
0812 06:56:27.488374 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:27.488385 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:27.488422 (+ 37us) server_negotiation.cc:407] Connection header received
0812 06:56:27.489982 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:27.490002 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:27.490331 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:27.490697 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:27.492141 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.493441 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.494696 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.495322 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.495493 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:27.499181 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:27.499188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:27.499200 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:27.499233 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:27.502047 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.502713 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.502722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.502729 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.502937 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.503378 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.503381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.503383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.503964 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:27.504258 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:27.504608 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 06:56:27.504996 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":311,"threads_started":1}
I20260812 06:56:27.509930 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:27.513625 22359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } }
I20260812 06:56:27.514025 22359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:27.514158 22359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cea77b49bd9448a8dcbf60c7ac1f93c, State: Initialized, Role: FOLLOWER
I20260812 06:56:27.514745 22359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } }
I20260812 06:56:27.515035 22359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:27.515177 22359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:27.515326 22359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:27.517827 22359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } }
I20260812 06:56:27.518267 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cea77b49bd9448a8dcbf60c7ac1f93c; no voters:
I20260812 06:56:27.518918 22359 leader_election.cc:288] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:27.519009 22373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:27.520152 22373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [term 1 LEADER]: Becoming Leader. State: Replica: 3cea77b49bd9448a8dcbf60c7ac1f93c, State: Running, Role: LEADER
I20260812 06:56:27.520660 22373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } }
I20260812 06:56:27.521178 22359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:27.525808 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } } }
I20260812 06:56:27.527485 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:27.526539 22379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cea77b49bd9448a8dcbf60c7ac1f93c. Latest consensus state: current_term: 1 leader_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cea77b49bd9448a8dcbf60c7ac1f93c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38665 } } }
I20260812 06:56:27.533948 22379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:27.534377 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:27.513908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43717 (local address 127.0.0.1:38665)
0812 06:56:27.514091 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:27.514096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:27.514525 (+ 429us) server_negotiation.cc:407] Connection header received
0812 06:56:27.514673 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:27.514679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:27.514755 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:27.514862 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:27.516259 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.517317 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.518554 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:27.519326 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:27.522371 (+ 3045us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:27.522585 (+ 214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:27.522590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:27.522594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:27.522622 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:27.530178 (+ 7556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.531161 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.531167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.531171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.531330 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:27.531810 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:27.531815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:27.531819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:27.532215 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:27.533521 (+ 1306us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:27.533672 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 06:56:27.533906 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 06:56:27.540520 22385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:27.545835 22385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:27.554435 22385 catalog_manager.cc:1347] Generated new cluster ID: 7cbd9b0813a647718c4e13deb70eb054
I20260812 06:56:27.554534 22385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:27.569494 22385 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:27.571143 22385 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:27.588075 22385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cea77b49bd9448a8dcbf60c7ac1f93c: Generated new TSK 0
I20260812 06:56:27.588815 22385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:27.593663 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22215
I20260812 06:56:27.594820 22346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:27.683033 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22215
2026-08-12T06:56:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:27 dist-test-slave-6zbq krb5kdc[22411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:27 dist-test-slave-6zbq krb5kdc[22411](info): set up 2 sockets
Aug 12 06:56:27 dist-test-slave-6zbq krb5kdc[22411](info): commencing operation
krb5kdc: starting...
W20260812 06:56:28.982781 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:29 dist-test-slave-6zbq krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:29.239255 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:29.552146 22445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:29.552397 22445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:29.552505 22445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:29.559399 22445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:29.559512 22445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:29.559579 22445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:29.559653 22445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:29.559698 22445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:29.568924 22445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:29.569945 22445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:29.571149 22445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:29.581597 22452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.581692 22454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:29.581652 22451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:29.582739 22445 server_base.cc:1034] running on GCE node
I20260812 06:56:29.583410 22445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:29.584595 22445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:29.585781 22445 hybrid_clock.cc:648] HybridClock initialized: now 1786517789585740 us; error 29 us; skew 500 ppm
I20260812 06:56:29.586076 22445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:29 dist-test-slave-6zbq krb5kdc[22411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:29.588966 22445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:29.590219 22445 webserver.cc:460] Webserver started at http://127.0.0.1:36087/ using document root <none> and password file <none>
I20260812 06:56:29.590811 22445 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:29.590909 22445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:29.591141 22445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:29.593824 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "df09456206ab41668ed6b82856bb07fe"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-6zbq"
server_key: "5efd2860c029f278e1293bf03a756a9e"
server_key_iv: "9a6feb4269f51066c93a82490ec079a9"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.594563 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "df09456206ab41668ed6b82856bb07fe"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-6zbq"
server_key: "5efd2860c029f278e1293bf03a756a9e"
server_key_iv: "9a6feb4269f51066c93a82490ec079a9"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.599184 22445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:29.602317 22471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:29.603538 22445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:29.603699 22445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "df09456206ab41668ed6b82856bb07fe"
format_stamp: "Formatted at 2026-08-12 06:56:29 on dist-test-slave-6zbq"
server_key: "5efd2860c029f278e1293bf03a756a9e"
server_key_iv: "9a6feb4269f51066c93a82490ec079a9"
server_key_version: "encryptionkey@0"
I20260812 06:56:29.603864 22445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:29.616690 22445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:29.620083 22445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:29.620311 22445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:29.636015 22445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40743
I20260812 06:56:29.636051 22557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40743 every 8 connection(s)
I20260812 06:56:29.637151 22445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:29.639354 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22445
I20260812 06:56:29.639513 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:29.639912 17685 external_mini_cluster.cc:1428] Setting key 74d7024aea03d852cb0311da105f40b4
I20260812 06:56:29.640532 22558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:29.653172 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.641530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34595 (local address 127.0.0.1:40743)
0812 06:56:29.642303 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.642318 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.642390 (+ 72us) server_negotiation.cc:407] Connection header received
0812 06:56:29.643959 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.643985 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.644450 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.644858 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.645332 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.647266 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.648505 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.649476 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.649644 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.650629 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.650640 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.650656 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:29.651342 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.651428 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:29.651467 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.651577 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.651742 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.651987 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":327,"threads_started":1}
I20260812 06:56:29.654719 22558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe: Bootstrap starting.
I20260812 06:56:29.656680 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:29.657852 22558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:29.658818 22558 log.cc:824] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:29.661625 22558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe: No bootstrap required, opened a new log
I20260812 06:56:29.667281 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:29.660889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53367 (local address 127.0.0.1:40743)
0812 06:56:29.661051 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:29.661056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:29.661713 (+ 657us) server_negotiation.cc:407] Connection header received
0812 06:56:29.661872 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:29.661878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:29.661946 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:29.662058 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:29.662526 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.663594 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.664804 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:29.665492 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:29.665615 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:29.666597 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:29.666603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:29.666608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:29.666627 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:29.666678 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:29.666702 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:29.666792 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:29.666869 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 06:56:29.666955 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:29.670145 22558 raft_consensus.cc:348] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } }
I20260812 06:56:29.670518 22558 raft_consensus.cc:374] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:29.670614 22558 raft_consensus.cc:729] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df09456206ab41668ed6b82856bb07fe, State: Initialized, Role: FOLLOWER
I20260812 06:56:29.671110 22558 consensus_queue.cc:260] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } }
I20260812 06:56:29.671310 22558 raft_consensus.cc:388] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:29.671438 22558 raft_consensus.cc:482] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:29.671587 22558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:29.673653 22558 raft_consensus.cc:504] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } }
I20260812 06:56:29.674101 22558 leader_election.cc:302] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df09456206ab41668ed6b82856bb07fe; no voters:
I20260812 06:56:29.674633 22558 leader_election.cc:288] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:29.674728 22569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:29.675392 22569 raft_consensus.cc:686] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [term 1 LEADER]: Becoming Leader. State: Replica: df09456206ab41668ed6b82856bb07fe, State: Running, Role: LEADER
I20260812 06:56:29.675880 22569 consensus_queue.cc:237] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } }
I20260812 06:56:29.676432 22558 sys_catalog.cc:564] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:29.678382 22571 sys_catalog.cc:455] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df09456206ab41668ed6b82856bb07fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } } }
I20260812 06:56:29.678766 22571 sys_catalog.cc:458] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:29.679700 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader df09456206ab41668ed6b82856bb07fe. Latest consensus state: current_term: 1 leader_uuid: "df09456206ab41668ed6b82856bb07fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df09456206ab41668ed6b82856bb07fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40743 } } }
I20260812 06:56:29.681567 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:29.682363 22578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:29.687040 22578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:29.695820 22578 catalog_manager.cc:1347] Generated new cluster ID: 4086a26482054ffd9953f10955ac2eb6
I20260812 06:56:29.695919 22578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:29.704210 22578 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:29.705358 22578 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:29.713881 22578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df09456206ab41668ed6b82856bb07fe: Generated new TSK 0
I20260812 06:56:29.714582 22578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:29.726707 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22445
W20260812 06:56:29.727726 22547 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34595
W20260812 06:56:29.728317 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:29.728444 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22445
2026-08-12T06:56:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:29 dist-test-slave-6zbq krb5kdc[22609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:29 dist-test-slave-6zbq krb5kdc[22609](info): set up 2 sockets
Aug 12 06:56:29 dist-test-slave-6zbq krb5kdc[22609](info): commencing operation
krb5kdc: starting...
W20260812 06:56:30.968889 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:31 dist-test-slave-6zbq krb5kdc[22609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:31.252962 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:31.570046 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:31.570322 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:31.570477 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:31.577286 22634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:31.577389 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:31.577467 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:31.577550 22634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:31.577612 22634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:31.586752 22634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:31.587828 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:31.588981 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:31.599711 22644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:31.599804 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:31.599875 22652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:31.600692 22634 server_base.cc:1034] running on GCE node
I20260812 06:56:31.601390 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:31.602619 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:31.603816 22634 hybrid_clock.cc:648] HybridClock initialized: now 1786517791603764 us; error 42 us; skew 500 ppm
I20260812 06:56:31.604139 22634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:31 dist-test-slave-6zbq krb5kdc[22609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:31.610198 22634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:31.611485 22634 webserver.cc:460] Webserver started at http://127.0.0.1:41099/ using document root <none> and password file <none>
I20260812 06:56:31.612064 22634 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:31.612186 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:31.612452 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:31.615252 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "ea2c482ded47488d84865b921bfa0f02"
format_stamp: "Formatted at 2026-08-12 06:56:31 on dist-test-slave-6zbq"
server_key: "25ec40d440bcd854ba79c7bf8ee9a441"
server_key_iv: "cf91b489d03b5f3dbdfb3edeb870347f"
server_key_version: "encryptionkey@0"
I20260812 06:56:31.615952 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "ea2c482ded47488d84865b921bfa0f02"
format_stamp: "Formatted at 2026-08-12 06:56:31 on dist-test-slave-6zbq"
server_key: "25ec40d440bcd854ba79c7bf8ee9a441"
server_key_iv: "cf91b489d03b5f3dbdfb3edeb870347f"
server_key_version: "encryptionkey@0"
I20260812 06:56:31.620589 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:31.623695 22668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:31.624866 22634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:56:31.625020 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "ea2c482ded47488d84865b921bfa0f02"
format_stamp: "Formatted at 2026-08-12 06:56:31 on dist-test-slave-6zbq"
server_key: "25ec40d440bcd854ba79c7bf8ee9a441"
server_key_iv: "cf91b489d03b5f3dbdfb3edeb870347f"
server_key_version: "encryptionkey@0"
I20260812 06:56:31.625188 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:31.630486 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:31.633896 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:31.634141 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:31.650312 22634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38137
I20260812 06:56:31.650389 22758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38137 every 8 connection(s)
I20260812 06:56:31.651502 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:31.654736 22759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:31.654883 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22634
I20260812 06:56:31.655063 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:31.655459 17685 external_mini_cluster.cc:1428] Setting key 0fc66afe6a96f27e9053ed95a4c38e6b
I20260812 06:56:31.662262 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:31.656956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40697 (local address 127.0.0.1:38137)
0812 06:56:31.657737 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:31.657748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:31.657787 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:31.659275 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:31.659294 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:31.659582 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:31.659874 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:31.660872 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:31.660878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:31.660890 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:31.660904 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:31.660960 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:31.660978 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:31.661155 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:31.661268 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 06:56:31.661441 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":345,"threads_started":1}
I20260812 06:56:31.665630 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:31.667372 22759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02: Bootstrap starting.
I20260812 06:56:31.670872 22759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:31.671836 22759 log.cc:824] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:31.672761 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:31.670484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36619 (local address 127.0.0.1:38137)
0812 06:56:31.670638 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:31.670643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:31.671053 (+ 410us) server_negotiation.cc:407] Connection header received
0812 06:56:31.671223 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:31.671227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:31.671273 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:31.671347 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:31.672162 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:31.672168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:31.672172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:31.672190 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:31.672225 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:31.672243 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:31.672333 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:31.672448 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 06:56:31.672501 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:56:31.674566 22759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02: No bootstrap required, opened a new log
I20260812 06:56:31.681403 22759 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } }
I20260812 06:56:31.681737 22759 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:31.681828 22759 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2c482ded47488d84865b921bfa0f02, State: Initialized, Role: FOLLOWER
I20260812 06:56:31.682307 22759 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } }
I20260812 06:56:31.682554 22759 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:31.682689 22759 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:31.682847 22759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:31.685014 22759 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } }
I20260812 06:56:31.685439 22759 leader_election.cc:302] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea2c482ded47488d84865b921bfa0f02; no voters:
I20260812 06:56:31.685932 22759 leader_election.cc:288] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:31.686039 22781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:31.686663 22781 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [term 1 LEADER]: Becoming Leader. State: Replica: ea2c482ded47488d84865b921bfa0f02, State: Running, Role: LEADER
I20260812 06:56:31.687130 22781 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } }
I20260812 06:56:31.687732 22759 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:31.689585 22782 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea2c482ded47488d84865b921bfa0f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } } }
I20260812 06:56:31.689971 22782 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:31.690366 22783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea2c482ded47488d84865b921bfa0f02. Latest consensus state: current_term: 1 leader_uuid: "ea2c482ded47488d84865b921bfa0f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2c482ded47488d84865b921bfa0f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38137 } } }
I20260812 06:56:31.690728 22783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:31.692909 22792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:31.697430 22792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:31.705325 22792 catalog_manager.cc:1347] Generated new cluster ID: 0058226afe0241aabf7be38a434bd4d2
I20260812 06:56:31.705425 22792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:31.731560 22792 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:31.731894 22792 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:31.737109 22792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea2c482ded47488d84865b921bfa0f02: Generated new TSK 0
I20260812 06:56:31.737735 22792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:31.742218 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22634
W20260812 06:56:31.743582 22753 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40697
W20260812 06:56:31.744187 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:31.744304 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22634
2026-08-12T06:56:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:31 dist-test-slave-6zbq krb5kdc[22812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:31 dist-test-slave-6zbq krb5kdc[22812](info): set up 2 sockets
Aug 12 06:56:31 dist-test-slave-6zbq krb5kdc[22812](info): commencing operation
krb5kdc: starting...
W20260812 06:56:33.004045 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:33 dist-test-slave-6zbq krb5kdc[22812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:33.284667 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:33.600062 22830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:33.600313 22830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:33.600436 22830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:33.607321 22830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:33.607412 22830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:33.607486 22830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:33.607558 22830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:33.607611 22830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:33.616642 22830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:33.617693 22830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:33.618865 22830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:33.629565 22850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.629565 22846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:33.629774 22844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:33.630394 22830 server_base.cc:1034] running on GCE node
I20260812 06:56:33.631084 22830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:33.632269 22830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:33.633471 22830 hybrid_clock.cc:648] HybridClock initialized: now 1786517793633431 us; error 35 us; skew 500 ppm
I20260812 06:56:33.633781 22830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:33 dist-test-slave-6zbq krb5kdc[22812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:33.636662 22830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:33.637982 22830 webserver.cc:460] Webserver started at http://127.0.0.1:41395/ using document root <none> and password file <none>
I20260812 06:56:33.638629 22830 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:33.638748 22830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:33.639012 22830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:33.641714 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "be41df7bcab048198078e2dd42b2d7f6"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-6zbq"
server_key: "a21d42896de4d6b48a7f98091289eb6f"
server_key_iv: "1bf152f2301156dfd2963d5ae2881089"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.642470 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "be41df7bcab048198078e2dd42b2d7f6"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-6zbq"
server_key: "a21d42896de4d6b48a7f98091289eb6f"
server_key_iv: "1bf152f2301156dfd2963d5ae2881089"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.647089 22830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:33.650393 22860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:33.651562 22830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 06:56:33.651729 22830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "be41df7bcab048198078e2dd42b2d7f6"
format_stamp: "Formatted at 2026-08-12 06:56:33 on dist-test-slave-6zbq"
server_key: "a21d42896de4d6b48a7f98091289eb6f"
server_key_iv: "1bf152f2301156dfd2963d5ae2881089"
server_key_version: "encryptionkey@0"
I20260812 06:56:33.651896 22830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:33.690102 22830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:33.694202 22830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:33.694489 22830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:33.711387 22830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34385
I20260812 06:56:33.711428 22959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34385 every 8 connection(s)
I20260812 06:56:33.712641 22830 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:33.716089 22960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:33.716936 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 22830
I20260812 06:56:33.717128 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:33.717655 17685 external_mini_cluster.cc:1428] Setting key 883768a347cefc9ea055b22338a3c145
I20260812 06:56:33.729329 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6: Bootstrap starting.
Aug 12 06:56:33 dist-test-slave-6zbq krb5kdc[22812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:56:33.732918 22960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:33.733968 22960 log.cc:824] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:33.736742 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6: No bootstrap required, opened a new log
I20260812 06:56:33.740327 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.719441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:34161 (local address 127.0.0.1:34385)
0812 06:56:33.720155 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:33.720173 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:33.720215 (+ 42us) server_negotiation.cc:407] Connection header received
0812 06:56:33.722107 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:33.722129 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:33.722553 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:33.722868 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:33.724477 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.725875 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.727692 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.728318 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.728439 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.733289 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:33.733297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:33.733312 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:33.733348 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:33.736310 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.737170 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.737176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.737181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.737348 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.737806 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.737809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.737812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.738288 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:33.738659 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:33.739143 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 06:56:33.739419 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":313,"threads_started":1}
I20260812 06:56:33.743789 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:33.744611 22960 raft_consensus.cc:348] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } }
I20260812 06:56:33.744916 22960 raft_consensus.cc:374] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:33.745004 22960 raft_consensus.cc:729] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be41df7bcab048198078e2dd42b2d7f6, State: Initialized, Role: FOLLOWER
I20260812 06:56:33.745478 22960 consensus_queue.cc:260] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } }
I20260812 06:56:33.745690 22960 raft_consensus.cc:388] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:33.745806 22960 raft_consensus.cc:482] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:33.745951 22960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:33.748095 22960 raft_consensus.cc:504] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } }
I20260812 06:56:33.748579 22960 leader_election.cc:302] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be41df7bcab048198078e2dd42b2d7f6; no voters:
I20260812 06:56:33.749193 22960 leader_election.cc:288] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:33.749328 22980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:33.749938 22980 raft_consensus.cc:686] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [term 1 LEADER]: Becoming Leader. State: Replica: be41df7bcab048198078e2dd42b2d7f6, State: Running, Role: LEADER
I20260812 06:56:33.750590 22980 consensus_queue.cc:237] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } }
I20260812 06:56:33.752110 22980 sys_catalog.cc:455] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be41df7bcab048198078e2dd42b2d7f6. Latest consensus state: current_term: 1 leader_uuid: "be41df7bcab048198078e2dd42b2d7f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } } }
I20260812 06:56:33.752523 22980 sys_catalog.cc:458] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:33.752995 22983 sys_catalog.cc:455] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be41df7bcab048198078e2dd42b2d7f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be41df7bcab048198078e2dd42b2d7f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34385 } } }
I20260812 06:56:33.753389 22983 sys_catalog.cc:458] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:33.757661 22960 sys_catalog.cc:564] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:33.760106 22985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:33.765339 22985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:33.765604 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:33.748012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:45173 (local address 127.0.0.1:34385)
0812 06:56:33.748184 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:33.748190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:33.748553 (+ 363us) server_negotiation.cc:407] Connection header received
0812 06:56:33.748660 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:33.748665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:33.748750 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:33.748847 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:33.750637 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.751885 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.753621 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:33.754369 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:33.754486 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:33.757691 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:33.757698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:33.757702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:33.757746 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:33.761425 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.762383 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.762390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.762394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.762549 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:33.763128 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:33.763134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:33.763138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:33.763600 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:33.763896 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:33.764996 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0812 06:56:33.765187 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:56:33.774685 22985 catalog_manager.cc:1347] Generated new cluster ID: 22b2fe91f1ce42f4978f556df33144b7
I20260812 06:56:33.774787 22985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:33.802865 22985 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:33.804054 22985 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:33.809798 22985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be41df7bcab048198078e2dd42b2d7f6: Generated new TSK 0
I20260812 06:56:33.810745 22985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:33.820979 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 22830
I20260812 06:56:33.822556 22954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:33.910873 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 22830
W20260812 06:56:33.932624 22810 connection.cc:535] client connection to 127.0.0.1:34385 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34385): Connection reset by peer (error 104)
2026-08-12T06:56:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:34 dist-test-slave-6zbq krb5kdc[23014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:34 dist-test-slave-6zbq krb5kdc[23014](info): set up 2 sockets
Aug 12 06:56:34 dist-test-slave-6zbq krb5kdc[23014](info): commencing operation
krb5kdc: starting...
W20260812 06:56:35.168351 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:35 dist-test-slave-6zbq krb5kdc[23014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:35.423499 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:35.741941 23041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:35.742280 23041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:35.742439 23041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:35.749354 23041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:35.749445 23041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:35.749495 23041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:35.749552 23041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:35.749593 23041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:35.758924 23041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:35.759985 23041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:35.761147 23041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:35.771749 23050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.771816 23053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:35.771796 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:35.772444 23041 server_base.cc:1034] running on GCE node
I20260812 06:56:35.773124 23041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:35.774312 23041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:35.775533 23041 hybrid_clock.cc:648] HybridClock initialized: now 1786517795775488 us; error 36 us; skew 500 ppm
I20260812 06:56:35.775846 23041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:35 dist-test-slave-6zbq krb5kdc[23014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517795, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:35.778697 23041 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:35.779956 23041 webserver.cc:460] Webserver started at http://127.0.0.1:44061/ using document root <none> and password file <none>
I20260812 06:56:35.780522 23041 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:35.780637 23041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:35.780903 23041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:35.783649 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "1ad61f0fdd0847a98e963431b3ff6f62"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-6zbq"
server_key: "0d3ce54a8584a6a662ed76c2994af1c4"
server_key_iv: "1f26f104638daab587e775f82a901741"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.784358 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "1ad61f0fdd0847a98e963431b3ff6f62"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-6zbq"
server_key: "0d3ce54a8584a6a662ed76c2994af1c4"
server_key_iv: "1f26f104638daab587e775f82a901741"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.788950 23041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:35.792073 23069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:35.793241 23041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:56:35.793390 23041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "1ad61f0fdd0847a98e963431b3ff6f62"
format_stamp: "Formatted at 2026-08-12 06:56:35 on dist-test-slave-6zbq"
server_key: "0d3ce54a8584a6a662ed76c2994af1c4"
server_key_iv: "1f26f104638daab587e775f82a901741"
server_key_version: "encryptionkey@0"
I20260812 06:56:35.793555 23041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:35.824453 23041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:35.827884 23041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:35.828117 23041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:35.844430 23041 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40515
I20260812 06:56:35.844488 23165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40515 every 8 connection(s)
I20260812 06:56:35.845762 23041 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:35.849088 23166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:35.854779 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23041
I20260812 06:56:35.854964 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:35.855485 17685 external_mini_cluster.cc:1428] Setting key 2716cf60afae8c8c48c75ce8b360dbee
I20260812 06:56:35.859225 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62: Bootstrap starting.
I20260812 06:56:35.862267 23166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:35.863440 23166 log.cc:824] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:35.866379 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62: No bootstrap required, opened a new log
I20260812 06:56:35.868171 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.857151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:57685 (local address 127.0.0.1:40515)
0812 06:56:35.857854 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.857864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.857903 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:35.859550 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.859571 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.859905 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.860185 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.860770 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.861998 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.863777 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.864788 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.864907 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.866056 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.866064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.866076 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:35.866622 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.866704 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:35.866730 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.866852 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.867108 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.867278 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":318,"threads_started":1}
I20260812 06:56:35.871886 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:35.875902 23166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260812 06:56:35.876330 23166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:35.876452 23166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad61f0fdd0847a98e963431b3ff6f62, State: Initialized, Role: FOLLOWER
I20260812 06:56:35.877180 23166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260812 06:56:35.877437 23166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:35.877590 23166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:35.877784 23166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:35.880458 23166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260812 06:56:35.880951 23166 leader_election.cc:302] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ad61f0fdd0847a98e963431b3ff6f62; no voters:
I20260812 06:56:35.881601 23166 leader_election.cc:288] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:35.881693 23183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:35.883138 23183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [term 1 LEADER]: Becoming Leader. State: Replica: 1ad61f0fdd0847a98e963431b3ff6f62, State: Running, Role: LEADER
I20260812 06:56:35.883750 23183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260812 06:56:35.883946 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:35.877019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:37169 (local address 127.0.0.1:40515)
0812 06:56:35.877230 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:35.877236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:35.877537 (+ 301us) server_negotiation.cc:407] Connection header received
0812 06:56:35.877725 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:35.877730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:35.877811 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:35.877920 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:35.878485 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.879582 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.880868 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:35.881566 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:35.881702 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:35.882984 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:35.882991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:35.882995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:35.883019 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:35.883061 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:35.883083 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:35.883233 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:35.883503 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 06:56:35.883551 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:35.884017 23166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:35.886626 23186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } } }
I20260812 06:56:35.887037 23186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:35.887490 23187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ad61f0fdd0847a98e963431b3ff6f62. Latest consensus state: current_term: 1 leader_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad61f0fdd0847a98e963431b3ff6f62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } } }
I20260812 06:56:35.887789 23187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:35.888490 23195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:35.893446 23195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:35.902150 23195 catalog_manager.cc:1347] Generated new cluster ID: 6482e1fb81a54d889d2a52ab8e8bde63
I20260812 06:56:35.902299 23195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:35.919443 23195 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:35.920660 23195 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:35.929713 23195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ad61f0fdd0847a98e963431b3ff6f62: Generated new TSK 0
I20260812 06:56:35.930420 23195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:35.944154 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23041
W20260812 06:56:35.946017 23161 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:57685
W20260812 06:56:35.946719 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23041: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:35.946869 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23041
W20260812 06:56:35.976701 23012 connection.cc:535] client connection to 127.0.0.1:40515 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40515): Connection reset by peer (error 104)
2026-08-12T06:56:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:36 dist-test-slave-6zbq krb5kdc[23225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:36 dist-test-slave-6zbq krb5kdc[23225](info): set up 2 sockets
Aug 12 06:56:36 dist-test-slave-6zbq krb5kdc[23225](info): commencing operation
krb5kdc: starting...
W20260812 06:56:37.206990 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:37 dist-test-slave-6zbq krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:37.481966 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:37.801299 23253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:37.801594 23253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:37.801728 23253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:37.808761 23253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:37.808866 23253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:37.808943 23253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:37.809017 23253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:37.809075 23253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:37.818464 23253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:37.819577 23253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:37.820801 23253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:37.831976 23264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:37.831933 23268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:37.832005 23265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:37.832975 23253 server_base.cc:1034] running on GCE node
I20260812 06:56:37.833638 23253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:37.834918 23253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:37.836119 23253 hybrid_clock.cc:648] HybridClock initialized: now 1786517797836071 us; error 42 us; skew 500 ppm
I20260812 06:56:37.836422 23253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:37 dist-test-slave-6zbq krb5kdc[23225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:37.845252 23253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:37.846633 23253 webserver.cc:460] Webserver started at http://127.0.0.1:34197/ using document root <none> and password file <none>
I20260812 06:56:37.847208 23253 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:37.847304 23253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:37.847543 23253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:37.850238 23253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "2b6bd7a9d1034930a7e72a6c5f696695"
format_stamp: "Formatted at 2026-08-12 06:56:37 on dist-test-slave-6zbq"
server_key: "6ca9cc241893416ffd412b6906183b4d"
server_key_iv: "4dc62190b0704021ac756511482c4b6a"
server_key_version: "encryptionkey@0"
I20260812 06:56:37.850986 23253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "2b6bd7a9d1034930a7e72a6c5f696695"
format_stamp: "Formatted at 2026-08-12 06:56:37 on dist-test-slave-6zbq"
server_key: "6ca9cc241893416ffd412b6906183b4d"
server_key_iv: "4dc62190b0704021ac756511482c4b6a"
server_key_version: "encryptionkey@0"
I20260812 06:56:37.855633 23253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 06:56:37.858695 23279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:37.859843 23253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:37.859989 23253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "2b6bd7a9d1034930a7e72a6c5f696695"
format_stamp: "Formatted at 2026-08-12 06:56:37 on dist-test-slave-6zbq"
server_key: "6ca9cc241893416ffd412b6906183b4d"
server_key_iv: "4dc62190b0704021ac756511482c4b6a"
server_key_version: "encryptionkey@0"
I20260812 06:56:37.860149 23253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:37.865463 23253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:37.869240 23253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:37.869486 23253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:37.886101 23253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40123
I20260812 06:56:37.886164 23364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40123 every 8 connection(s)
I20260812 06:56:37.887295 23253 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:37.890643 23366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:37.892949 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23253
I20260812 06:56:37.893113 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:37.893518 17685 external_mini_cluster.cc:1428] Setting key 4683e60e32b96b45d76b01432c321167
I20260812 06:56:37.900559 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.895164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:58481 (local address 127.0.0.1:40123)
0812 06:56:37.895928 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.895957 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.895996 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:37.897568 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.897589 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.897897 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.898200 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.899176 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.899182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.899198 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:37.899213 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.899276 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:37.899305 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.899481 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.899584 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.899763 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":341,"threads_started":1}
I20260812 06:56:37.901715 23366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695: Bootstrap starting.
I20260812 06:56:37.903924 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:37.904810 23366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:37.905727 23366 log.cc:824] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:37.908623 23366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695: No bootstrap required, opened a new log
I20260812 06:56:37.910185 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:37.907917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40051 (local address 127.0.0.1:40123)
0812 06:56:37.908033 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:37.908036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:37.908546 (+ 510us) server_negotiation.cc:407] Connection header received
0812 06:56:37.908686 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:37.908690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:37.908733 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:37.908814 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:37.909594 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:37.909599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:37.909604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:37.909621 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:37.909654 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:37.909668 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:37.909757 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:37.909883 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 06:56:37.909939 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 06:56:37.916203 23366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260812 06:56:37.916505 23366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:37.916590 23366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6bd7a9d1034930a7e72a6c5f696695, State: Initialized, Role: FOLLOWER
I20260812 06:56:37.917070 23366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260812 06:56:37.917286 23366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:37.917408 23366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:37.917551 23366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:37.919926 23366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260812 06:56:37.920365 23366 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6bd7a9d1034930a7e72a6c5f696695; no voters:
I20260812 06:56:37.920855 23366 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:37.920955 23385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:37.921597 23385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6bd7a9d1034930a7e72a6c5f696695, State: Running, Role: LEADER
I20260812 06:56:37.922055 23385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260812 06:56:37.922665 23366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:37.927764 23391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260812 06:56:37.928191 23391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:37.928761 23396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6bd7a9d1034930a7e72a6c5f696695. Latest consensus state: current_term: 1 leader_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6bd7a9d1034930a7e72a6c5f696695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260812 06:56:37.930574 23396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:37.931778 23418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:37.936385 23418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:37.944314 23418 catalog_manager.cc:1347] Generated new cluster ID: 95b57a9ce06b43c9bd88510569016919
I20260812 06:56:37.944413 23418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:37.957861 23418 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:37.958217 23418 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:37.966694 23418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b6bd7a9d1034930a7e72a6c5f696695: Generated new TSK 0
I20260812 06:56:37.967455 23418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:37.984069 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23253
W20260812 06:56:37.985107 23359 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:58481
W20260812 06:56:37.985692 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23253: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:37.985806 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23253
2026-08-12T06:56:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:38 dist-test-slave-6zbq krb5kdc[23438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:38 dist-test-slave-6zbq krb5kdc[23438](info): set up 2 sockets
Aug 12 06:56:38 dist-test-slave-6zbq krb5kdc[23438](info): commencing operation
krb5kdc: starting...
W20260812 06:56:39.226292 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:39 dist-test-slave-6zbq krb5kdc[23438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:39.484869 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:39.797894 23456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:39.798148 23456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:39.798292 23456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:39.805141 23456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:39.805254 23456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:39.805330 23456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:39.805411 23456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:39.805470 23456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:39.815037 23456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:39.816097 23456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:39.817273 23456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:39.828049 23467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:39.828145 23466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:39.828133 23470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:39.831446 23456 server_base.cc:1034] running on GCE node
I20260812 06:56:39.832082 23456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:39.833233 23456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:39.834425 23456 hybrid_clock.cc:648] HybridClock initialized: now 1786517799834391 us; error 28 us; skew 500 ppm
I20260812 06:56:39.834705 23456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:39 dist-test-slave-6zbq krb5kdc[23438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517799, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:39.837545 23456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:39.838891 23456 webserver.cc:460] Webserver started at http://127.0.0.1:37395/ using document root <none> and password file <none>
I20260812 06:56:39.839495 23456 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:39.839607 23456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:39.839879 23456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:39.842674 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "128b2d52e39043cda5b8199eb2e0b577"
format_stamp: "Formatted at 2026-08-12 06:56:39 on dist-test-slave-6zbq"
server_key: "e5581a160aaa7bc1807bda8835860dfc"
server_key_iv: "2e5544c91e1b1562fbedef2bbb2e6bdd"
server_key_version: "encryptionkey@0"
I20260812 06:56:39.843392 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "128b2d52e39043cda5b8199eb2e0b577"
format_stamp: "Formatted at 2026-08-12 06:56:39 on dist-test-slave-6zbq"
server_key: "e5581a160aaa7bc1807bda8835860dfc"
server_key_iv: "2e5544c91e1b1562fbedef2bbb2e6bdd"
server_key_version: "encryptionkey@0"
I20260812 06:56:39.848031 23456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 06:56:39.851229 23479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:39.852356 23456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:39.852499 23456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "128b2d52e39043cda5b8199eb2e0b577"
format_stamp: "Formatted at 2026-08-12 06:56:39 on dist-test-slave-6zbq"
server_key: "e5581a160aaa7bc1807bda8835860dfc"
server_key_iv: "2e5544c91e1b1562fbedef2bbb2e6bdd"
server_key_version: "encryptionkey@0"
I20260812 06:56:39.852662 23456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:39.872588 23456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:39.876421 23456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:39.876657 23456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:39.892735 23456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33163
I20260812 06:56:39.892772 23593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33163 every 8 connection(s)
I20260812 06:56:39.893894 23456 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:39.895236 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23456
I20260812 06:56:39.895401 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:39.895814 17685 external_mini_cluster.cc:1428] Setting key cf72303c208051ebaa51f0a21fac27d6
I20260812 06:56:39.897626 23594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:56:39 dist-test-slave-6zbq krb5kdc[23438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 06:56:39.909906 23594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577: Bootstrap starting.
I20260812 06:56:39.912933 23594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:39.913991 23594 log.cc:824] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:39.916915 23594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577: No bootstrap required, opened a new log
I20260812 06:56:39.917631 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.897398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:41429 (local address 127.0.0.1:33163)
0812 06:56:39.898139 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.898153 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.898210 (+ 57us) server_negotiation.cc:407] Connection header received
0812 06:56:39.900325 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.900352 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.900815 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.901243 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:39.902486 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.904442 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.905588 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.906599 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.906773 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.910757 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:39.910765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:39.910777 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:39.910810 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:39.913613 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.914480 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.914489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.914500 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.914735 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.915168 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.915173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.915176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.915670 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:39.915887 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.916394 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.916644 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":297,"threads_started":1}
I20260812 06:56:39.921039 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:39.924367 23594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260812 06:56:39.924691 23594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:39.924782 23594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128b2d52e39043cda5b8199eb2e0b577, State: Initialized, Role: FOLLOWER
I20260812 06:56:39.925395 23594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260812 06:56:39.925606 23594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:39.925757 23594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:39.925930 23594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:39.928561 23594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260812 06:56:39.929086 23594 leader_election.cc:302] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 128b2d52e39043cda5b8199eb2e0b577; no voters:
I20260812 06:56:39.929663 23594 leader_election.cc:288] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:39.929754 23606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:39.931607 23594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:39.934610 23606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [term 1 LEADER]: Becoming Leader. State: Replica: 128b2d52e39043cda5b8199eb2e0b577, State: Running, Role: LEADER
I20260812 06:56:39.935185 23606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260812 06:56:39.939528 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128b2d52e39043cda5b8199eb2e0b577. Latest consensus state: current_term: 1 leader_uuid: "128b2d52e39043cda5b8199eb2e0b577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
I20260812 06:56:39.939898 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:39.940419 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "128b2d52e39043cda5b8199eb2e0b577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128b2d52e39043cda5b8199eb2e0b577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
I20260812 06:56:39.940704 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:39.942219 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:39.924958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:52637 (local address 127.0.0.1:33163)
0812 06:56:39.925119 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:39.925124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:39.925621 (+ 497us) server_negotiation.cc:407] Connection header received
0812 06:56:39.925771 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:39.925777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:39.925836 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:39.925929 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:39.927703 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.928393 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.930193 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:39.930866 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:39.930999 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:39.933932 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:39.933939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:39.933944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:39.933986 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:39.937834 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.938865 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.938871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.938875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.939096 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:39.939679 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:39.939684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:39.939687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:39.940091 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:39.940434 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:39.941677 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0812 06:56:39.941865 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 06:56:39.943907 23628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:39.948731 23628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:39.956061 23628 catalog_manager.cc:1347] Generated new cluster ID: 0bb051f22c00490db45a6286b99baf92
I20260812 06:56:39.956177 23628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:39.975281 23628 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:39.976881 23628 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:39.988776 23628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 128b2d52e39043cda5b8199eb2e0b577: Generated new TSK 0
I20260812 06:56:39.989643 23628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:39.998194 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23456
I20260812 06:56:39.999385 23585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:40.088249 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23456
W20260812 06:56:40.109301 23433 connection.cc:535] client connection to 127.0.0.1:33163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33163): Connection reset by peer (error 104)
2026-08-12T06:56:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2097 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:40 dist-test-slave-6zbq krb5kdc[23648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:40 dist-test-slave-6zbq krb5kdc[23648](info): set up 2 sockets
Aug 12 06:56:40 dist-test-slave-6zbq krb5kdc[23648](info): commencing operation
krb5kdc: starting...
W20260812 06:56:41.343326 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:41 dist-test-slave-6zbq krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:41.610695 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:41.924741 23676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:41.925010 23676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:41.925158 23676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:41.931977 23676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:41.932080 23676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:41.932164 23676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:41.932246 23676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:41.932312 23676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:41.941427 23676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:41.942548 23676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:41.943764 23676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:41.954401 23690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:41.954330 23691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:41.954298 23693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:41.955343 23676 server_base.cc:1034] running on GCE node
I20260812 06:56:41.955998 23676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:41.957180 23676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:41.958391 23676 hybrid_clock.cc:648] HybridClock initialized: now 1786517801958346 us; error 35 us; skew 500 ppm
I20260812 06:56:41.958689 23676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:41 dist-test-slave-6zbq krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517801, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:41.961573 23676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:41.962934 23676 webserver.cc:460] Webserver started at http://127.0.0.1:45137/ using document root <none> and password file <none>
I20260812 06:56:41.963521 23676 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:41.963624 23676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:41.963893 23676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:41.966648 23676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "455eea2563b04e919dd99888ded6b5b8"
format_stamp: "Formatted at 2026-08-12 06:56:41 on dist-test-slave-6zbq"
server_key: "f8249004ef7ce4db2dd11ad7050486d0"
server_key_iv: "8cb74b622349bcc9f92cc980ea996fde"
server_key_version: "encryptionkey@0"
I20260812 06:56:41.967351 23676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "455eea2563b04e919dd99888ded6b5b8"
format_stamp: "Formatted at 2026-08-12 06:56:41 on dist-test-slave-6zbq"
server_key: "f8249004ef7ce4db2dd11ad7050486d0"
server_key_iv: "8cb74b622349bcc9f92cc980ea996fde"
server_key_version: "encryptionkey@0"
I20260812 06:56:41.971930 23676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:41.975018 23715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:41.976166 23676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:41.976310 23676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "455eea2563b04e919dd99888ded6b5b8"
format_stamp: "Formatted at 2026-08-12 06:56:41 on dist-test-slave-6zbq"
server_key: "f8249004ef7ce4db2dd11ad7050486d0"
server_key_iv: "8cb74b622349bcc9f92cc980ea996fde"
server_key_version: "encryptionkey@0"
I20260812 06:56:41.976469 23676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:41.993387 23676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:41.997154 23676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:41.997395 23676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:42.013801 23676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42763
I20260812 06:56:42.013839 23813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42763 every 8 connection(s)
I20260812 06:56:42.015002 23676 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:42.018271 23815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:42.021425 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23676
I20260812 06:56:42.021601 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:42.022187 17685 external_mini_cluster.cc:1428] Setting key d20eba2ec556cef107fb30fd2f2eacfa
I20260812 06:56:42.029842 23815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8: Bootstrap starting.
I20260812 06:56:42.033136 23815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:42.034217 23815 log.cc:824] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:42.034641 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:42.023845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:34743 (local address 127.0.0.1:42763)
0812 06:56:42.024510 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:42.024521 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:42.024559 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:56:42.026142 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:42.026170 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:42.026590 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:42.026893 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:42.027475 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.028772 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.030507 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.031138 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.031294 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:42.032502 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:42.032509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:42.032521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:42.032996 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:42.033073 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:42.033099 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:42.033236 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:42.033505 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 06:56:42.033672 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":295,"threads_started":1}
I20260812 06:56:42.037256 23815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8: No bootstrap required, opened a new log
I20260812 06:56:42.038539 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:42.045676 23815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260812 06:56:42.046031 23815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:42.046128 23815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455eea2563b04e919dd99888ded6b5b8, State: Initialized, Role: FOLLOWER
I20260812 06:56:42.046746 23815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260812 06:56:42.047039 23815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:42.047190 23815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:42.047366 23815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:42.049077 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:42.042812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54589 (local address 127.0.0.1:42763)
0812 06:56:42.042956 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:42.042960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:42.043210 (+ 250us) server_negotiation.cc:407] Connection header received
0812 06:56:42.043340 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:42.043346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:42.043426 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:42.043536 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:42.043966 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.045096 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.046543 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:42.047201 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:42.047331 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:42.048383 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:42.048390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:42.048394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:42.048418 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:42.048462 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:42.048481 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:42.048593 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:42.048718 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 06:56:42.048772 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 06:56:42.050318 23815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260812 06:56:42.050825 23815 leader_election.cc:302] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 455eea2563b04e919dd99888ded6b5b8; no voters:
I20260812 06:56:42.051376 23815 leader_election.cc:288] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:42.051510 23834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:42.052172 23834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [term 1 LEADER]: Becoming Leader. State: Replica: 455eea2563b04e919dd99888ded6b5b8, State: Running, Role: LEADER
I20260812 06:56:42.052663 23834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } }
I20260812 06:56:42.053236 23815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:42.055197 23835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "455eea2563b04e919dd99888ded6b5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } } }
I20260812 06:56:42.055544 23835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:42.056818 23846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:42.057029 23837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 455eea2563b04e919dd99888ded6b5b8. Latest consensus state: current_term: 1 leader_uuid: "455eea2563b04e919dd99888ded6b5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455eea2563b04e919dd99888ded6b5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42763 } } }
I20260812 06:56:42.057422 23837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:42.062484 23846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:42.070636 23846 catalog_manager.cc:1347] Generated new cluster ID: 0779a73917e24208832aa0e17b0931ad
I20260812 06:56:42.070740 23846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:42.091928 23846 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:42.093187 23846 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:42.113339 23846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 455eea2563b04e919dd99888ded6b5b8: Generated new TSK 0
I20260812 06:56:42.114287 23846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:42.131803 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23676
W20260812 06:56:42.133203 23803 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:34743
W20260812 06:56:42.133857 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23676: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:42.134009 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23676
W20260812 06:56:42.155228 23641 connection.cc:535] client connection to 127.0.0.1:42763 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42763): Connection reset by peer (error 104)
2026-08-12T06:56:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:42 dist-test-slave-6zbq krb5kdc[23879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:42 dist-test-slave-6zbq krb5kdc[23879](info): set up 2 sockets
Aug 12 06:56:42 dist-test-slave-6zbq krb5kdc[23879](info): commencing operation
krb5kdc: starting...
W20260812 06:56:43.384225 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:43 dist-test-slave-6zbq krb5kdc[23879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 06:56:43.735889 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:44.270515 23898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:44.270968 23898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:44.271203 23898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:44.282523 23898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:44.282691 23898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:44.282814 23898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:44.282963 23898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:44.283059 23898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:44.298105 23898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:44.300051 23898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:44.302101 23898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:44.320425 23910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:44.320530 23919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:44.320495 23911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:44.322016 23898 server_base.cc:1034] running on GCE node
I20260812 06:56:44.323365 23898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:44.326123 23898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:44.327486 23898 hybrid_clock.cc:648] HybridClock initialized: now 1786517804327412 us; error 65 us; skew 500 ppm
I20260812 06:56:44.328039 23898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:44 dist-test-slave-6zbq krb5kdc[23879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517804, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:44.332922 23898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 06:56:44.335093 23898 webserver.cc:460] Webserver started at http://127.0.0.1:39787/ using document root <none> and password file <none>
I20260812 06:56:44.336095 23898 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:44.336329 23898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:44.336800 23898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:44.341711 23898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "3477459832c14122b41936eb1bbf8a38"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-6zbq"
server_key: "cbca2da47a5a89e3fa88d2188e68e272"
server_key_iv: "6b2422caee3d932bbda79276d76dc56b"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.342996 23898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "3477459832c14122b41936eb1bbf8a38"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-6zbq"
server_key: "cbca2da47a5a89e3fa88d2188e68e272"
server_key_iv: "6b2422caee3d932bbda79276d76dc56b"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.351397 23898 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260812 06:56:44.356760 23929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:44.358862 23898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 06:56:44.359127 23898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "3477459832c14122b41936eb1bbf8a38"
format_stamp: "Formatted at 2026-08-12 06:56:44 on dist-test-slave-6zbq"
server_key: "cbca2da47a5a89e3fa88d2188e68e272"
server_key_iv: "6b2422caee3d932bbda79276d76dc56b"
server_key_version: "encryptionkey@0"
I20260812 06:56:44.359411 23898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:44.368402 23898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:44.374827 23898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:44.375267 23898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:44.402207 23898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39241
I20260812 06:56:44.402310 24018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39241 every 8 connection(s)
I20260812 06:56:44.404250 23898 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:44.405866 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 23898
I20260812 06:56:44.406212 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:44.407269 17685 external_mini_cluster.cc:1428] Setting key e1e0078e5070a3c9d0a2f832a442c858
I20260812 06:56:44.411449 24019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:44.419518 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.410112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54219 (local address 127.0.0.1:39241)
0812 06:56:44.411299 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.411324 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.411395 (+ 71us) server_negotiation.cc:407] Connection header received
0812 06:56:44.414105 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.414139 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.414677 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.415232 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.416818 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:44.416826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:44.416849 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:44.416889 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:44.417009 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:44.417045 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:44.417309 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:44.417552 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 06:56:44.417896 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":524,"threads_started":1}
I20260812 06:56:44.425467 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:44.432499 24019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38: Bootstrap starting.
I20260812 06:56:44.437767 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:44.434046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:60219 (local address 127.0.0.1:39241)
0812 06:56:44.434334 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:44.434389 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:44.434990 (+ 601us) server_negotiation.cc:407] Connection header received
0812 06:56:44.435216 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:44.435221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:44.435331 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:44.435530 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:44.436683 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:44.436700 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:44.436707 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:44.436740 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:44.436832 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:44.436877 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:44.437070 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:44.437162 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 06:56:44.437263 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 06:56:44.438189 24019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:44.440276 24019 log.cc:824] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:44.444394 24019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38: No bootstrap required, opened a new log
I20260812 06:56:44.456429 24019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 06:56:44.457012 24019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:44.457152 24019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3477459832c14122b41936eb1bbf8a38, State: Initialized, Role: FOLLOWER
I20260812 06:56:44.457962 24019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 06:56:44.458312 24019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:44.458590 24019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:44.458844 24019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:44.462328 24019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 06:56:44.463073 24019 leader_election.cc:302] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3477459832c14122b41936eb1bbf8a38; no voters:
I20260812 06:56:44.463857 24019 leader_election.cc:288] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:44.464003 24040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:44.465169 24040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [term 1 LEADER]: Becoming Leader. State: Replica: 3477459832c14122b41936eb1bbf8a38, State: Running, Role: LEADER
I20260812 06:56:44.466166 24040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 06:56:44.467271 24019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:44.470695 24042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3477459832c14122b41936eb1bbf8a38. Latest consensus state: current_term: 1 leader_uuid: "3477459832c14122b41936eb1bbf8a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260812 06:56:44.471338 24042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:44.472460 24041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3477459832c14122b41936eb1bbf8a38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3477459832c14122b41936eb1bbf8a38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260812 06:56:44.472997 24041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:44.474273 24051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:44.481819 24051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:44.494189 24051 catalog_manager.cc:1347] Generated new cluster ID: da023d53aba343d699323a031813e2a2
I20260812 06:56:44.494383 24051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:44.528635 24051 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:44.529258 24051 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:44.544760 24051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3477459832c14122b41936eb1bbf8a38: Generated new TSK 0
I20260812 06:56:44.545840 24051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:44.551716 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23898
W20260812 06:56:44.553506 24011 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:54219
W20260812 06:56:44.554502 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 23898: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:44.554754 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 23898
2026-08-12T06:56:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:44 dist-test-slave-6zbq krb5kdc[24081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:44 dist-test-slave-6zbq krb5kdc[24081](info): set up 2 sockets
Aug 12 06:56:44 dist-test-slave-6zbq krb5kdc[24081](info): commencing operation
krb5kdc: starting...
W20260812 06:56:46.445314 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.827s user 0.003s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:46 dist-test-slave-6zbq krb5kdc[24081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:46.710860 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38593
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:47.036401 24097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:47.036652 24097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:47.036789 24097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:47.043802 24097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:47.043917 24097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:47.043995 24097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:47.044075 24097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:47.044139 24097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:47.053568 24097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:47.054745 24097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:47.056015 24097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:47.067142 24112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:47.067091 24115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:47.067184 24119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:47.067894 24097 server_base.cc:1034] running on GCE node
I20260812 06:56:47.068573 24097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:47.069816 24097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:47.071036 24097 hybrid_clock.cc:648] HybridClock initialized: now 1786517807071005 us; error 81 us; skew 500 ppm
I20260812 06:56:47.071369 24097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:47 dist-test-slave-6zbq krb5kdc[24081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517807, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:47.074200 24097 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:47.075546 24097 webserver.cc:460] Webserver started at http://127.17.69.126:36057/ using document root <none> and password file <none>
I20260812 06:56:47.076154 24097 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:47.076274 24097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:47.076541 24097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:47.079368 24097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "85a034e62b8d47fc87973d1ac05fbfd3"
format_stamp: "Formatted at 2026-08-12 06:56:47 on dist-test-slave-6zbq"
server_key: "2258f04de34972d4f1e05aff8064a990"
server_key_iv: "2956a926227f0b36dcc0a69c567e1aca"
server_key_version: "encryptionkey@0"
I20260812 06:56:47.080088 24097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "85a034e62b8d47fc87973d1ac05fbfd3"
format_stamp: "Formatted at 2026-08-12 06:56:47 on dist-test-slave-6zbq"
server_key: "2258f04de34972d4f1e05aff8064a990"
server_key_iv: "2956a926227f0b36dcc0a69c567e1aca"
server_key_version: "encryptionkey@0"
I20260812 06:56:47.084811 24097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:47.088115 24133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:47.089581 24097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 06:56:47.089771 24097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "85a034e62b8d47fc87973d1ac05fbfd3"
format_stamp: "Formatted at 2026-08-12 06:56:47 on dist-test-slave-6zbq"
server_key: "2258f04de34972d4f1e05aff8064a990"
server_key_iv: "2956a926227f0b36dcc0a69c567e1aca"
server_key_version: "encryptionkey@0"
I20260812 06:56:47.089953 24097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:47.115069 24097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:47.118892 24097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:47.119136 24097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:47.136142 24097 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38593
I20260812 06:56:47.136175 24209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38593 every 8 connection(s)
I20260812 06:56:47.137364 24097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:47.140736 24210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:47.142822 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24097
I20260812 06:56:47.143007 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:47.143517 17685 external_mini_cluster.cc:1428] Setting key 0872da67c96358fedbca70d5aa4e83ba
I20260812 06:56:47.152750 24210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3: Bootstrap starting.
I20260812 06:56:47.156453 24210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:47.157622 24210 log.cc:824] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3: Log is configured to *not* fsync() on all Append() calls
Aug 12 06:56:47 dist-test-slave-6zbq krb5kdc[24081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:47.160881 24210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3: No bootstrap required, opened a new log
I20260812 06:56:47.167335 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.145189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49439 (local address 127.17.69.126:38593)
0812 06:56:47.145948 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.145962 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.146000 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:56:47.147676 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.147696 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.148043 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.148325 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:47.150037 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.151451 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.153267 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.153924 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.154045 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.158965 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:47.158976 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:47.158994 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:47.159042 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:47.162803 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.163517 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.163525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.163533 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.163760 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.164278 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.164284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.164287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.164970 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:47.165304 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.165655 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.165997 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":346,"threads_started":1}
I20260812 06:56:47.171092 24210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } }
I20260812 06:56:47.171525 24210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:47.171649 24210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a034e62b8d47fc87973d1ac05fbfd3, State: Initialized, Role: FOLLOWER
I20260812 06:56:47.171985 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:47.172243 24210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } }
I20260812 06:56:47.172469 24210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:47.172612 24210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:47.172757 24210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:47.174885 24210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } }
I20260812 06:56:47.175335 24210 leader_election.cc:302] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85a034e62b8d47fc87973d1ac05fbfd3; no voters:
I20260812 06:56:47.175923 24210 leader_election.cc:288] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:47.176038 24226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:47.176868 24226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [term 1 LEADER]: Becoming Leader. State: Replica: 85a034e62b8d47fc87973d1ac05fbfd3, State: Running, Role: LEADER
I20260812 06:56:47.177508 24226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } }
I20260812 06:56:47.179412 24226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85a034e62b8d47fc87973d1ac05fbfd3. Latest consensus state: current_term: 1 leader_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } } }
I20260812 06:56:47.180982 24226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:47.180605 24229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85a034e62b8d47fc87973d1ac05fbfd3" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38593 } } }
I20260812 06:56:47.181461 24229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:47.184067 24210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:47.184274 24231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:47.189743 24231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:47.193595 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:47.176144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38261 (local address 127.17.69.126:38593)
0812 06:56:47.176321 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:47.176328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:47.176719 (+ 391us) server_negotiation.cc:407] Connection header received
0812 06:56:47.176988 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:47.176993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:47.177065 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:47.177128 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:47.178797 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.179963 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.181688 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:47.182465 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:47.182600 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:47.185752 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:47.185759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:47.185764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:47.185807 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:47.189436 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.190422 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.190427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.190431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.190582 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:47.191141 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:47.191146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:47.191149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:47.191566 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:47.191830 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:47.193023 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0812 06:56:47.193248 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 06:56:47.199627 24231 catalog_manager.cc:1347] Generated new cluster ID: 3be41d8a4c3a440b9071725a8adce2c9
I20260812 06:56:47.199733 24231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:47.238297 24231 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:47.240116 24231 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:47.253888 24231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85a034e62b8d47fc87973d1ac05fbfd3: Generated new TSK 0
I20260812 06:56:47.254650 24231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:47.270035 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24097
I20260812 06:56:47.271570 24199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:47.363461 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24097
W20260812 06:56:47.389799 24077 connection.cc:535] client connection to 127.17.69.126:38593 recv error: Network error: failed to read from TLS socket (remote: 127.17.69.126:38593): Connection reset by peer (error 104)
2026-08-12T06:56:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2794 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:47 dist-test-slave-6zbq krb5kdc[24281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:47 dist-test-slave-6zbq krb5kdc[24281](info): set up 2 sockets
Aug 12 06:56:47 dist-test-slave-6zbq krb5kdc[24281](info): commencing operation
krb5kdc: starting...
W20260812 06:56:48.641393 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:48 dist-test-slave-6zbq krb5kdc[24281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:48.949262 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38675
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:49.288105 24306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:49.288362 24306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:49.288458 24306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:49.295327 24306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:49.295419 24306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:49.295471 24306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:49.295528 24306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:49.295571 24306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:49.304812 24306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:49.305886 24306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:49.307102 24306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:49.318025 24320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:49.317998 24318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:49.318451 24317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:49.318877 24306 server_base.cc:1034] running on GCE node
I20260812 06:56:49.319569 24306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:49.320812 24306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:49.322013 24306 hybrid_clock.cc:648] HybridClock initialized: now 1786517809321971 us; error 31 us; skew 500 ppm
I20260812 06:56:49.322327 24306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:49 dist-test-slave-6zbq krb5kdc[24281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517809, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:49.328442 24306 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:49.329869 24306 webserver.cc:460] Webserver started at http://127.17.69.126:43519/ using document root <none> and password file <none>
I20260812 06:56:49.330559 24306 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:49.330679 24306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:49.330955 24306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:49.333755 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "46fead7469f742d1ad2c7f11d903a697"
format_stamp: "Formatted at 2026-08-12 06:56:49 on dist-test-slave-6zbq"
server_key: "8ad535cbef9d6980233afb5dfa3bc01b"
server_key_iv: "f41f843d0541d2ad10ee58fae270ace8"
server_key_version: "encryptionkey@0"
I20260812 06:56:49.334519 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "46fead7469f742d1ad2c7f11d903a697"
format_stamp: "Formatted at 2026-08-12 06:56:49 on dist-test-slave-6zbq"
server_key: "8ad535cbef9d6980233afb5dfa3bc01b"
server_key_iv: "f41f843d0541d2ad10ee58fae270ace8"
server_key_version: "encryptionkey@0"
I20260812 06:56:49.339432 24306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 06:56:49.342728 24331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:49.343896 24306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 06:56:49.344051 24306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "46fead7469f742d1ad2c7f11d903a697"
format_stamp: "Formatted at 2026-08-12 06:56:49 on dist-test-slave-6zbq"
server_key: "8ad535cbef9d6980233afb5dfa3bc01b"
server_key_iv: "f41f843d0541d2ad10ee58fae270ace8"
server_key_version: "encryptionkey@0"
I20260812 06:56:49.344218 24306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:49.357622 24306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:49.361518 24306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:49.361769 24306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:49.378976 24306 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38675
I20260812 06:56:49.379000 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38675 every 8 connection(s)
I20260812 06:56:49.380259 24306 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:49.383688 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:49.389067 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24306
I20260812 06:56:49.389217 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:49.389621 17685 external_mini_cluster.cc:1428] Setting key a0ff1fe1c5b743aa0910d177d011ea31
I20260812 06:56:49.394616 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697: Bootstrap starting.
I20260812 06:56:49.398443 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:49.399426 24425 log.cc:824] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:49.402585 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697: No bootstrap required, opened a new log
I20260812 06:56:49.404198 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:49.391342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55787 (local address 127.17.69.126:38675)
0812 06:56:49.392149 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:49.392163 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:49.392214 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:56:49.394476 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:49.394503 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:49.394957 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:49.395367 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:49.395827 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:49.397917 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:49.399246 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:49.400203 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:49.400334 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:49.401585 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:49.401595 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:49.401612 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:49.402285 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:49.402394 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:49.402428 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:49.402600 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:49.402760 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 06:56:49.402914 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":328,"threads_started":1}
I20260812 06:56:49.407750 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:49.410491 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } }
I20260812 06:56:49.410887 24425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:49.410983 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46fead7469f742d1ad2c7f11d903a697, State: Initialized, Role: FOLLOWER
I20260812 06:56:49.411516 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } }
I20260812 06:56:49.411747 24425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:49.411877 24425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:49.412088 24425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:49.415135 24425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } }
I20260812 06:56:49.415593 24425 leader_election.cc:302] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46fead7469f742d1ad2c7f11d903a697; no voters:
I20260812 06:56:49.416227 24425 leader_election.cc:288] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:49.416340 24438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:49.417338 24438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [term 1 LEADER]: Becoming Leader. State: Replica: 46fead7469f742d1ad2c7f11d903a697, State: Running, Role: LEADER
I20260812 06:56:49.417946 24438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } }
I20260812 06:56:49.418480 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:49.418977 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:49.412169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39569 (local address 127.17.69.126:38675)
0812 06:56:49.412340 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:49.412346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:49.412691 (+ 345us) server_negotiation.cc:407] Connection header received
0812 06:56:49.412832 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:49.412837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:49.412921 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:49.413038 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:49.413492 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:49.414656 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:49.415994 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:49.416538 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:49.416689 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:49.418217 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:49.418224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:49.418228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:49.418249 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:49.418294 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:49.418312 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:49.418436 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:49.418618 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 06:56:49.418668 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:56:49.424141 24441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46fead7469f742d1ad2c7f11d903a697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } } }
I20260812 06:56:49.424185 24442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46fead7469f742d1ad2c7f11d903a697. Latest consensus state: current_term: 1 leader_uuid: "46fead7469f742d1ad2c7f11d903a697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46fead7469f742d1ad2c7f11d903a697" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38675 } } }
I20260812 06:56:49.424580 24442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:49.424577 24441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:49.425952 24454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:49.430317 24454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:49.437826 24454 catalog_manager.cc:1347] Generated new cluster ID: 77f98232edc64aeb874d752022cd6f2f
I20260812 06:56:49.437923 24454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:49.457283 24454 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:49.458758 24454 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:49.468853 24454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46fead7469f742d1ad2c7f11d903a697: Generated new TSK 0
I20260812 06:56:49.469871 24454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:49.488077 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24306
W20260812 06:56:49.489048 24416 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55787
W20260812 06:56:49.489715 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24306: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:49.489845 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24306
W20260812 06:56:49.516448 24276 connection.cc:535] client connection to 127.17.69.126:38675 recv error: Network error: failed to read from TLS socket (remote: 127.17.69.126:38675): Connection reset by peer (error 104)
2026-08-12T06:56:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2126 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:49 dist-test-slave-6zbq krb5kdc[24478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:49 dist-test-slave-6zbq krb5kdc[24478](info): set up 2 sockets
Aug 12 06:56:49 dist-test-slave-6zbq krb5kdc[24478](info): commencing operation
krb5kdc: starting...
W20260812 06:56:50.752672 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:50 dist-test-slave-6zbq krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:51.021200 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46287
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:46287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:51.347458 24508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:51.347728 24508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:51.347857 24508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:51.354786 24508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:51.354883 24508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:51.354957 24508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:51.355032 24508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:51.355098 24508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:51.364411 24508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:46287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:46287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:51.365571 24508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:51.366808 24508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:51.377761 24518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:51.377835 24522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:51.377805 24525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:51.378538 24508 server_base.cc:1034] running on GCE node
I20260812 06:56:51.379257 24508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:51.380493 24508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:51.381686 24508 hybrid_clock.cc:648] HybridClock initialized: now 1786517811381640 us; error 43 us; skew 500 ppm
I20260812 06:56:51.381987 24508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:51 dist-test-slave-6zbq krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517811, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:51.384887 24508 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:51.386186 24508 webserver.cc:460] Webserver started at http://127.17.69.126:35031/ using document root <none> and password file <none>
I20260812 06:56:51.386838 24508 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:51.386957 24508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:51.387245 24508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:51.390019 24508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "48b26128152f4c02a9b7f23047c16315"
format_stamp: "Formatted at 2026-08-12 06:56:51 on dist-test-slave-6zbq"
server_key: "e243aa644898d23be6ccfde4d93cbda0"
server_key_iv: "538dd7adeda360fd5a11ce31bc6bd687"
server_key_version: "encryptionkey@0"
I20260812 06:56:51.390759 24508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "48b26128152f4c02a9b7f23047c16315"
format_stamp: "Formatted at 2026-08-12 06:56:51 on dist-test-slave-6zbq"
server_key: "e243aa644898d23be6ccfde4d93cbda0"
server_key_iv: "538dd7adeda360fd5a11ce31bc6bd687"
server_key_version: "encryptionkey@0"
I20260812 06:56:51.395433 24508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:56:51.398530 24537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:51.399654 24508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:56:51.399808 24508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "48b26128152f4c02a9b7f23047c16315"
format_stamp: "Formatted at 2026-08-12 06:56:51 on dist-test-slave-6zbq"
server_key: "e243aa644898d23be6ccfde4d93cbda0"
server_key_iv: "538dd7adeda360fd5a11ce31bc6bd687"
server_key_version: "encryptionkey@0"
I20260812 06:56:51.399982 24508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:51.405385 24508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:51.408910 24508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:51.409173 24508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:51.425901 24508 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:46287
I20260812 06:56:51.425935 24621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:46287 every 8 connection(s)
I20260812 06:56:51.427382 24508 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:51.430823 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:51.434217 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24508
I20260812 06:56:51.434432 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:51.434973 17685 external_mini_cluster.cc:1428] Setting key c869804e62b2f811cce6d7cef316978a
I20260812 06:56:51.442204 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:51.436686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57127 (local address 127.17.69.126:46287)
0812 06:56:51.437433 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:51.437444 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:51.437484 (+ 40us) server_negotiation.cc:407] Connection header received
0812 06:56:51.439104 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:51.439127 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:51.439460 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:51.439760 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:51.440766 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:51.440775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:51.440797 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:51.440811 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:51.440868 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:51.440888 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:51.441055 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:51.441165 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 06:56:51.441345 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":335,"threads_started":1}
I20260812 06:56:51.442940 24622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315: Bootstrap starting.
I20260812 06:56:51.446038 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:51.446730 24622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:51.447678 24622 log.cc:824] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:51.450654 24622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315: No bootstrap required, opened a new log
I20260812 06:56:51.452466 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:51.450104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48663 (local address 127.17.69.126:46287)
0812 06:56:51.450240 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:51.450244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:51.450720 (+ 476us) server_negotiation.cc:407] Connection header received
0812 06:56:51.450869 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:51.450873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:51.450918 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:51.450992 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:51.451849 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:51.451856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:51.451861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:51.451878 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:51.451917 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:51.451935 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:51.452040 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:51.452149 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 06:56:51.452211 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 06:56:51.458505 24622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } }
I20260812 06:56:51.458832 24622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:51.458917 24622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b26128152f4c02a9b7f23047c16315, State: Initialized, Role: FOLLOWER
I20260812 06:56:51.459445 24622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } }
I20260812 06:56:51.459666 24622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:51.459789 24622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:51.459956 24622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:51.461959 24622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } }
I20260812 06:56:51.462448 24622 leader_election.cc:302] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48b26128152f4c02a9b7f23047c16315; no voters:
I20260812 06:56:51.462976 24622 leader_election.cc:288] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:51.463083 24639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:51.464416 24639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [term 1 LEADER]: Becoming Leader. State: Replica: 48b26128152f4c02a9b7f23047c16315, State: Running, Role: LEADER
I20260812 06:56:51.464824 24622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:51.464879 24639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } }
I20260812 06:56:51.467119 24641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48b26128152f4c02a9b7f23047c16315. Latest consensus state: current_term: 1 leader_uuid: "48b26128152f4c02a9b7f23047c16315" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } } }
I20260812 06:56:51.467440 24641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:51.467209 24640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48b26128152f4c02a9b7f23047c16315" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b26128152f4c02a9b7f23047c16315" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 46287 } } }
I20260812 06:56:51.467552 24640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:51.468380 24649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:51.473320 24649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:51.481701 24649 catalog_manager.cc:1347] Generated new cluster ID: a373180db93645afa2e162169f4d1c54
I20260812 06:56:51.481803 24649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:51.490810 24649 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:51.491216 24649 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:51.507283 24649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48b26128152f4c02a9b7f23047c16315: Generated new TSK 0
I20260812 06:56:51.507902 24649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:51.519872 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24508
W20260812 06:56:51.521064 24615 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57127
W20260812 06:56:51.521695 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:51.521833 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24508
2026-08-12T06:56:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:51 dist-test-slave-6zbq krb5kdc[24670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:51 dist-test-slave-6zbq krb5kdc[24670](info): set up 2 sockets
Aug 12 06:56:51 dist-test-slave-6zbq krb5kdc[24670](info): commencing operation
krb5kdc: starting...
W20260812 06:56:52.769891 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.008s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:52 dist-test-slave-6zbq krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:53.034552 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37429
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:35737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:53.370371 24692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:53.370613 24692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:53.370744 24692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:53.377574 24692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:53.377676 24692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:53.377754 24692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:53.377836 24692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:53.377893 24692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:53.386977 24692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:35737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:53.388020 24692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:53.389174 24692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:53.399624 24702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:53.399709 24704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:53.399675 24706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:53.400460 24692 server_base.cc:1034] running on GCE node
I20260812 06:56:53.401122 24692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:53.402376 24692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:53.403587 24692 hybrid_clock.cc:648] HybridClock initialized: now 1786517813403535 us; error 40 us; skew 500 ppm
I20260812 06:56:53.403892 24692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:53 dist-test-slave-6zbq krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517813, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:53.406759 24692 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:53.408108 24692 webserver.cc:460] Webserver started at http://127.17.69.126:45697/ using document root <none> and password file <none>
I20260812 06:56:53.408713 24692 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:53.408833 24692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:53.409106 24692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:53.411840 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "d5871c0dfb604a48b9472485eefa2006"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-6zbq"
server_key: "d2bfbfe5d1872c7a29ee4dfc4d2e7359"
server_key_iv: "bf0a2344a95ea4ebf8e4650abf937949"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.412554 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "d5871c0dfb604a48b9472485eefa2006"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-6zbq"
server_key: "d2bfbfe5d1872c7a29ee4dfc4d2e7359"
server_key_iv: "bf0a2344a95ea4ebf8e4650abf937949"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.417192 24692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 06:56:53.420346 24716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:53.421507 24692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 06:56:53.421674 24692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "d5871c0dfb604a48b9472485eefa2006"
format_stamp: "Formatted at 2026-08-12 06:56:53 on dist-test-slave-6zbq"
server_key: "d2bfbfe5d1872c7a29ee4dfc4d2e7359"
server_key_iv: "bf0a2344a95ea4ebf8e4650abf937949"
server_key_version: "encryptionkey@0"
I20260812 06:56:53.421835 24692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:53.436509 24692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:53.439936 24692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:53.440174 24692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:53.456591 24692 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:37429
I20260812 06:56:53.456614 24809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:37429 every 8 connection(s)
I20260812 06:56:53.457724 24692 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:53.458628 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24692
I20260812 06:56:53.458854 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:53.459372 17685 external_mini_cluster.cc:1428] Setting key f89595cffbad065003c467d667045973
I20260812 06:56:53.461776 24812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 06:56:53 dist-test-slave-6zbq krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:53.473763 24812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006: Bootstrap starting.
I20260812 06:56:53.476569 24812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:53.477496 24812 log.cc:824] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:53.480005 24812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006: No bootstrap required, opened a new log
I20260812 06:56:53.480599 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.460968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33311 (local address 127.17.69.126:37429)
0812 06:56:53.461752 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:53.461766 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:53.461804 (+ 38us) server_negotiation.cc:407] Connection header received
0812 06:56:53.463397 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:53.463419 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:53.463754 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:53.464049 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:53.465748 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.467111 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.468880 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.469484 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.469606 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.474043 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:53.474051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:53.474064 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:53.474102 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:53.476898 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.477536 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.477543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.477550 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.477739 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.478090 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.478095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.478098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.478621 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:53.478869 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:53.479314 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 06:56:53.479674 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":383,"threads_started":1}
I20260812 06:56:53.483891 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:53.487402 24812 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } }
I20260812 06:56:53.487705 24812 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:53.487804 24812 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5871c0dfb604a48b9472485eefa2006, State: Initialized, Role: FOLLOWER
I20260812 06:56:53.488364 24812 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } }
I20260812 06:56:53.488639 24812 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:53.488790 24812 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:53.488967 24812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:53.491590 24812 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } }
I20260812 06:56:53.492095 24812 leader_election.cc:302] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5871c0dfb604a48b9472485eefa2006; no voters:
I20260812 06:56:53.492687 24812 leader_election.cc:288] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:53.492767 24825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:53.493832 24825 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [term 1 LEADER]: Becoming Leader. State: Replica: d5871c0dfb604a48b9472485eefa2006, State: Running, Role: LEADER
I20260812 06:56:53.494397 24825 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } }
I20260812 06:56:53.494596 24812 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:53.497112 24827 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5871c0dfb604a48b9472485eefa2006. Latest consensus state: current_term: 1 leader_uuid: "d5871c0dfb604a48b9472485eefa2006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } } }
I20260812 06:56:53.497484 24827 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:53.498049 24826 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5871c0dfb604a48b9472485eefa2006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5871c0dfb604a48b9472485eefa2006" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37429 } } }
I20260812 06:56:53.498359 24826 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:53.498636 24836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:53.504091 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:53.487852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41357 (local address 127.17.69.126:37429)
0812 06:56:53.487968 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:53.487972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:53.488640 (+ 668us) server_negotiation.cc:407] Connection header received
0812 06:56:53.488782 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:53.488786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:53.488839 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:53.488930 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:53.490665 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.491343 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.493001 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:53.493729 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:53.493870 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:53.496792 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:53.496796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:53.496799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:53.496827 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:53.500073 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.500927 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.500932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.500936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.501139 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:53.501619 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:53.501624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:53.501628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:53.502003 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:53.503319 (+ 1316us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:53.503466 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 06:56:53.503685 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 06:56:53.509922 24836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:53.517840 24836 catalog_manager.cc:1347] Generated new cluster ID: 395c54688668435b9cd0f2ec989889c1
I20260812 06:56:53.517939 24836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:53.588837 24836 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:53.590637 24836 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:53.602384 24836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5871c0dfb604a48b9472485eefa2006: Generated new TSK 0
I20260812 06:56:53.603116 24836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:53.607874 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24692
I20260812 06:56:53.608879 24798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:56:53.700865 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24692
2026-08-12T06:56:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:53 dist-test-slave-6zbq krb5kdc[24854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:53 dist-test-slave-6zbq krb5kdc[24854](info): set up 2 sockets
Aug 12 06:56:53 dist-test-slave-6zbq krb5kdc[24854](info): commencing operation
krb5kdc: starting...
W20260812 06:56:54.939054 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:55 dist-test-slave-6zbq krb5kdc[24854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:55.221218 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:32977
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:55.541535 24880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:55.541803 24880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:55.541903 24880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:55.548914 24880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:55.549029 24880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:55.549077 24880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:55.549136 24880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:55.549180 24880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:55.559151 24880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:32977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:55.560457 24880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:55.561740 24880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:55.572652 24899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.572619 24894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:55.572696 24895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:55.574059 24880 server_base.cc:1034] running on GCE node
I20260812 06:56:55.574798 24880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:55.576048 24880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:55.577252 24880 hybrid_clock.cc:648] HybridClock initialized: now 1786517815577213 us; error 27 us; skew 500 ppm
I20260812 06:56:55.577555 24880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:55 dist-test-slave-6zbq krb5kdc[24854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517815, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:55.583871 24880 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:55.585201 24880 webserver.cc:460] Webserver started at http://127.17.69.126:43551/ using document root <none> and password file <none>
I20260812 06:56:55.585793 24880 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:55.585892 24880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:55.586134 24880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:55.588963 24880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "0b6d97e7b8514f2ca4063e2010586f4e"
format_stamp: "Formatted at 2026-08-12 06:56:55 on dist-test-slave-6zbq"
server_key: "889acfd657744753c89314e3a316958a"
server_key_iv: "4796edeb3609c10a16af7bb7bfffa082"
server_key_version: "encryptionkey@0"
I20260812 06:56:55.589665 24880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "0b6d97e7b8514f2ca4063e2010586f4e"
format_stamp: "Formatted at 2026-08-12 06:56:55 on dist-test-slave-6zbq"
server_key: "889acfd657744753c89314e3a316958a"
server_key_iv: "4796edeb3609c10a16af7bb7bfffa082"
server_key_version: "encryptionkey@0"
I20260812 06:56:55.594477 24880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 06:56:55.597570 24912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:55.598769 24880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:56:55.598908 24880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "0b6d97e7b8514f2ca4063e2010586f4e"
format_stamp: "Formatted at 2026-08-12 06:56:55 on dist-test-slave-6zbq"
server_key: "889acfd657744753c89314e3a316958a"
server_key_iv: "4796edeb3609c10a16af7bb7bfffa082"
server_key_version: "encryptionkey@0"
I20260812 06:56:55.599061 24880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:55.619649 24880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:55.623350 24880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:55.623585 24880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:55.640550 24880 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:32977
I20260812 06:56:55.640614 24999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:32977 every 8 connection(s)
I20260812 06:56:55.641816 24880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:55.645298 25002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:55.645970 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 24880
I20260812 06:56:55.646147 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:55.646747 17685 external_mini_cluster.cc:1428] Setting key a2b0e5fc7d5e6d79e2b93ec9893cbfa0
I20260812 06:56:55.658061 25002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e: Bootstrap starting.
I20260812 06:56:55.659487 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:55.648431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58525 (local address 127.17.69.126:32977)
0812 06:56:55.649154 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:55.649165 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:55.649204 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:55.651220 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:55.651242 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:55.651600 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:55.651907 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:55.652475 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.653669 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.655433 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.656098 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.656230 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:55.657391 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:55.657398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:55.657410 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:55.657895 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:55.657964 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:55.657987 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:55.658076 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:55.658315 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 06:56:55.658589 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":326,"threads_started":1}
I20260812 06:56:55.661096 25002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:55.662165 25002 log.cc:824] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:55.662897 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:55.664672 25002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e: No bootstrap required, opened a new log
I20260812 06:56:55.673192 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:55.667085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51901 (local address 127.17.69.126:32977)
0812 06:56:55.667274 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:55.667279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:55.667610 (+ 331us) server_negotiation.cc:407] Connection header received
0812 06:56:55.667783 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:55.667786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:55.667847 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:55.667918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:55.668494 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.669181 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.670998 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:55.671446 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:55.671537 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:55.672593 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:55.672597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:55.672600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:55.672619 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:55.672657 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:55.672672 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:55.672735 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:55.672867 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 06:56:55.672932 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:56:55.673534 25002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } }
I20260812 06:56:55.673918 25002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:55.674018 25002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6d97e7b8514f2ca4063e2010586f4e, State: Initialized, Role: FOLLOWER
I20260812 06:56:55.674613 25002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } }
I20260812 06:56:55.674834 25002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:55.674957 25002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:55.675091 25002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:55.677337 25002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } }
I20260812 06:56:55.677776 25002 leader_election.cc:302] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b6d97e7b8514f2ca4063e2010586f4e; no voters:
I20260812 06:56:55.678269 25002 leader_election.cc:288] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:55.678406 25016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:55.679033 25016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [term 1 LEADER]: Becoming Leader. State: Replica: 0b6d97e7b8514f2ca4063e2010586f4e, State: Running, Role: LEADER
I20260812 06:56:55.679533 25016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } }
I20260812 06:56:55.680128 25002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:55.682051 25019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b6d97e7b8514f2ca4063e2010586f4e. Latest consensus state: current_term: 1 leader_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } } }
I20260812 06:56:55.682449 25019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:55.683840 25018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6d97e7b8514f2ca4063e2010586f4e" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 32977 } } }
I20260812 06:56:55.684608 25018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:55.685120 25027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:55.690092 25027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:55.698763 25027 catalog_manager.cc:1347] Generated new cluster ID: 0919675c3d7c49508a4f9ecf2f884534
I20260812 06:56:55.698872 25027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:55.727862 25027 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:55.729219 25027 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:55.742646 25027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b6d97e7b8514f2ca4063e2010586f4e: Generated new TSK 0
I20260812 06:56:55.743872 25027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:55.759928 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24880
W20260812 06:56:55.760931 24993 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58525
W20260812 06:56:55.761626 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 24880: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:55.761756 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 24880
2026-08-12T06:56:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2056 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:55 dist-test-slave-6zbq krb5kdc[25058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:55 dist-test-slave-6zbq krb5kdc[25058](info): set up 2 sockets
Aug 12 06:56:55 dist-test-slave-6zbq krb5kdc[25058](info): commencing operation
krb5kdc: starting...
W20260812 06:56:57.015165 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:57 dist-test-slave-6zbq krb5kdc[25058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:57.311350 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45919
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:42819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:45919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:57.632836 25086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:57.633092 25086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:57.633246 25086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:57.640318 25086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:57.640436 25086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:57.640523 25086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:57.640614 25086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:57.640676 25086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:57.650022 25086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:42819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:45919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:45919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:57.651185 25086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:57.652429 25086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:57.663229 25098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:57.663257 25104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:57.663184 25101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:57.663959 25086 server_base.cc:1034] running on GCE node
I20260812 06:56:57.664705 25086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:57.665956 25086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:57.667145 25086 hybrid_clock.cc:648] HybridClock initialized: now 1786517817667101 us; error 32 us; skew 500 ppm
I20260812 06:56:57.667462 25086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:57 dist-test-slave-6zbq krb5kdc[25058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517817, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:57.670264 25086 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:57.671636 25086 webserver.cc:460] Webserver started at http://127.17.69.126:39903/ using document root <none> and password file <none>
I20260812 06:56:57.672245 25086 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:57.672359 25086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:57.672637 25086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:57.675451 25086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "025f92142ed14b858cff586faca28375"
format_stamp: "Formatted at 2026-08-12 06:56:57 on dist-test-slave-6zbq"
server_key: "0d4fe5ba7880dfa78b356e232b98666d"
server_key_iv: "e792ddc0feba4ee94b2f438408e41ef5"
server_key_version: "encryptionkey@0"
I20260812 06:56:57.676167 25086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "025f92142ed14b858cff586faca28375"
format_stamp: "Formatted at 2026-08-12 06:56:57 on dist-test-slave-6zbq"
server_key: "0d4fe5ba7880dfa78b356e232b98666d"
server_key_iv: "e792ddc0feba4ee94b2f438408e41ef5"
server_key_version: "encryptionkey@0"
I20260812 06:56:57.680843 25086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:56:57.684118 25115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:57.685297 25086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 06:56:57.685442 25086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "025f92142ed14b858cff586faca28375"
format_stamp: "Formatted at 2026-08-12 06:56:57 on dist-test-slave-6zbq"
server_key: "0d4fe5ba7880dfa78b356e232b98666d"
server_key_iv: "e792ddc0feba4ee94b2f438408e41ef5"
server_key_version: "encryptionkey@0"
I20260812 06:56:57.685611 25086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:57.690939 25086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:57.694644 25086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:57.694900 25086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:57.711585 25086 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:45919
I20260812 06:56:57.711637 25197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:45919 every 8 connection(s)
I20260812 06:56:57.712788 25086 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:57.716010 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25086
I20260812 06:56:57.716192 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:57.716301 25201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:57.716751 17685 external_mini_cluster.cc:1428] Setting key 2765cf9052aaf58da11f440901b24c47
I20260812 06:56:57.723737 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.718413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45019 (local address 127.17.69.126:45919)
0812 06:56:57.719177 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.719187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.719226 (+ 39us) server_negotiation.cc:407] Connection header received
0812 06:56:57.720772 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.720791 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.721100 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.721391 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.722313 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:57.722319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:57.722331 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:57.722373 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:57.722441 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:57.722465 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:57.722635 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:57.722747 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 06:56:57.722903 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":350,"threads_started":1}
I20260812 06:56:57.726883 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:56:57.729204 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375: Bootstrap starting.
I20260812 06:56:57.732637 25201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375: Neither blocks nor log segments found. Creating new log.
I20260812 06:56:57.733729 25201 log.cc:824] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375: Log is configured to *not* fsync() on all Append() calls
I20260812 06:56:57.733812 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:57.731626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55073 (local address 127.17.69.126:45919)
0812 06:56:57.731785 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:57.731790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:57.732225 (+ 435us) server_negotiation.cc:407] Connection header received
0812 06:56:57.732339 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:57.732343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:57.732394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:57.732494 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:57.733236 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:57.733241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:57.733244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 06:56:57.733261 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:57.733301 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 06:56:57.733319 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:57.733436 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:57.733503 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 06:56:57.733557 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:56:57.736321 25201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375: No bootstrap required, opened a new log
I20260812 06:56:57.743130 25201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } }
I20260812 06:56:57.743445 25201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:56:57.743528 25201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025f92142ed14b858cff586faca28375, State: Initialized, Role: FOLLOWER
I20260812 06:56:57.744037 25201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } }
I20260812 06:56:57.744269 25201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:56:57.744386 25201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:56:57.744524 25201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:56:57.746680 25201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } }
I20260812 06:56:57.747107 25201 leader_election.cc:302] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 025f92142ed14b858cff586faca28375; no voters:
I20260812 06:56:57.747623 25201 leader_election.cc:288] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 06:56:57.747727 25216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:56:57.748314 25216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [term 1 LEADER]: Becoming Leader. State: Replica: 025f92142ed14b858cff586faca28375, State: Running, Role: LEADER
I20260812 06:56:57.748829 25216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } }
I20260812 06:56:57.749382 25201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:56:57.751471 25217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "025f92142ed14b858cff586faca28375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } } }
I20260812 06:56:57.751962 25217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:57.751485 25220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 025f92142ed14b858cff586faca28375. Latest consensus state: current_term: 1 leader_uuid: "025f92142ed14b858cff586faca28375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "025f92142ed14b858cff586faca28375" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 45919 } } }
I20260812 06:56:57.752117 25220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375 [sys.catalog]: This master's current role is: LEADER
I20260812 06:56:57.755489 25234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:56:57.760143 25234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:56:57.767657 25234 catalog_manager.cc:1347] Generated new cluster ID: b26ee9c365884d819723b307a0bad678
I20260812 06:56:57.767741 25234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:56:57.775374 25234 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:56:57.775781 25234 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:56:57.783485 25234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 025f92142ed14b858cff586faca28375: Generated new TSK 0
I20260812 06:56:57.784140 25234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:56:57.790957 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25086
W20260812 06:56:57.791999 25191 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45019
W20260812 06:56:57.792708 17685 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25086: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 06:56:57.792847 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25086
2026-08-12T06:56:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2041 ms)
[----------] 18 tests from AuthTokenIssuingTest (38578 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:56:57 dist-test-slave-6zbq krb5kdc[25253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:56:57 dist-test-slave-6zbq krb5kdc[25253](info): set up 2 sockets
Aug 12 06:56:57 dist-test-slave-6zbq krb5kdc[25253](info): commencing operation
krb5kdc: starting...
W20260812 06:56:59.043286 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.011s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:56:59 dist-test-slave-6zbq krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:56:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:56:59.331199 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38903
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:34541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:56:59.651335 25278 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:56:59.651615 25278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:56:59.651685 25278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:56:59.651765 25278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:56:59.658625 25278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:56:59.658696 25278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:56:59.658766 25278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:56:59.658814 25278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:56:59.658885 25278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:56:59.668157 25278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:34541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:56:59.669225 25278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:56:59.670434 25278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:56:59.681905 25286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:59.681874 25287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:56:59.681962 25290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:56:59.682869 25278 server_base.cc:1034] running on GCE node
I20260812 06:56:59.683565 25278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:56:59.684772 25278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:56:59.685987 25278 hybrid_clock.cc:648] HybridClock initialized: now 1786517819685926 us; error 31 us; skew 500 ppm
I20260812 06:56:59.686280 25278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:56:59 dist-test-slave-6zbq krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:56:59.689113 25278 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:56:59.690412 25278 webserver.cc:460] Webserver started at http://127.17.69.126:36989/ using document root <none> and password file <none>
I20260812 06:56:59.691020 25278 fs_manager.cc:359] Metadata directory not provided
I20260812 06:56:59.691119 25278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:56:59.691414 25278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:56:59.694152 25278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "b7562b47719d47b2b036bc1e4f831401"
format_stamp: "Formatted at 2026-08-12 06:56:59 on dist-test-slave-6zbq"
server_key: "685eec45a760f33e7b89a464fb51f12d"
server_key_iv: "bc2777c66d725d08e6cba46dd0fe4552"
server_key_version: "encryptionkey@0"
I20260812 06:56:59.694947 25278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "b7562b47719d47b2b036bc1e4f831401"
format_stamp: "Formatted at 2026-08-12 06:56:59 on dist-test-slave-6zbq"
server_key: "685eec45a760f33e7b89a464fb51f12d"
server_key_iv: "bc2777c66d725d08e6cba46dd0fe4552"
server_key_version: "encryptionkey@0"
I20260812 06:56:59.699555 25278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 06:56:59.702757 25302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:56:59.703910 25278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260812 06:56:59.704061 25278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "b7562b47719d47b2b036bc1e4f831401"
format_stamp: "Formatted at 2026-08-12 06:56:59 on dist-test-slave-6zbq"
server_key: "685eec45a760f33e7b89a464fb51f12d"
server_key_iv: "bc2777c66d725d08e6cba46dd0fe4552"
server_key_version: "encryptionkey@0"
I20260812 06:56:59.704254 25278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:56:59.751080 25278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:56:59.754750 25278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:56:59.754976 25278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:56:59.771124 25278 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:38903
I20260812 06:56:59.771162 25398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:38903 every 8 connection(s)
I20260812 06:56:59.772267 25278 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:56:59.775597 25399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:56:59.777520 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25278
I20260812 06:56:59.777719 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:56:59.778321 17685 external_mini_cluster.cc:1428] Setting key 4274c66f8d4ad91451a38e4ed17bdb07
I20260812 06:56:59.781858 25399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:56:59 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:56:59.804497 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.784741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.17.69.126:38903)
0812 06:56:59.786270 (+ 1529us) server_negotiation.cc:206] Beginning negotiation
0812 06:56:59.786278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:56:59.786329 (+ 51us) server_negotiation.cc:407] Connection header received
0812 06:56:59.787815 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:56:59.787822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:56:59.788096 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:56:59.788301 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:56:59.789480 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.791133 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.792570 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.793331 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.793513 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.797304 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:56:59.797313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:56:59.797326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:56:59.797363 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:56:59.800101 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:59.800955 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:59.800962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:59.800964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:59.801157 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:56:59.801660 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:56:59.801664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:56:59.801666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:56:59.802227 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:56:59.802417 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 06:56:59.803152 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 06:56:59.803408 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":1320,"spinlock_wait_cycles":23296,"thread_start_us":1122,"threads_started":1}
I20260812 06:56:59.804553 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.784742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38903 (local address 127.0.0.1:54600)
0812 06:56:59.785567 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.785637 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 06:56:59.787449 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:56:59.788349 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:56:59.788355 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:56:59.788912 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:56:59.789269 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.789288 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.791319 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.791323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.792418 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:56:59.792431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:56:59.793558 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:56:59.793566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:56:59.793802 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:56:59.794515 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:56:59.794536 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:56:59.797043 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:56:59.800261 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:59.800273 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:59.800285 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:59.800799 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:59.801288 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:56:59.801293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:56:59.801297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:56:59.801540 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:56:59.802397 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:56:59.802405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:56:59.802712 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 06:56:59.802991 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 06:56:59.803568 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":124,"thread_start_us":344,"threads_started":1}
I20260812 06:56:59.807420 25399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:56:59.808552 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.808022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60624)
0812 06:56:59.808298 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.808423 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260812 06:56:59.808800 25303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
I20260812 06:56:59.810398 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.809946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60634)
0812 06:56:59.810094 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.810178 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:56:59.810662 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:36151: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.69.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
I20260812 06:56:59.846117 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:36151
--webserver_interface=127.17.69.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:34541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:56:59.859956 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } attempt: 1
I20260812 06:56:59.861055 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.860514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60638)
0812 06:56:59.860727 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.860890 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 06:56:59.862085 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.861685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60650)
0812 06:56:59.861837 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.861938 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:56:59.862277 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:36151: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
I20260812 06:56:59.930537 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } attempt: 2
I20260812 06:56:59.931679 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.931148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60658)
0812 06:56:59.931392 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.931501 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:56:59.932682 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:56:59.932285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60660)
0812 06:56:59.932435 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 06:56:59.932520 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 06:56:59.932864 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:36151: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
I20260812 06:57:00.024124 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } attempt: 3
I20260812 06:57:00.025293 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.024787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60662)
0812 06:57:00.025005 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.025125 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 06:57:00.026376 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.025930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60674)
0812 06:57:00.026082 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.026170 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:57:00.026607 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:36151: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
W20260812 06:57:00.162546 25408 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:57:00.162832 25408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:00.162904 25408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:00.162983 25408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:00.169786 25408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:00.169888 25408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:00.169970 25408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260812 06:57:00.169860 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } attempt: 4
W20260812 06:57:00.170039 25408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:00.170120 25408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:00.171118 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.170537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60678)
0812 06:57:00.170847 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.170953 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260812 06:57:00.172093 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.171712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60692)
0812 06:57:00.171848 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.171947 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 06:57:00.172288 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:36151: Network error: Client connection negotiation failed: client connection to 127.17.69.125:36151: connect: Connection refused (error 111)
I20260812 06:57:00.179844 25408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:34541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:36151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.69.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:57:00.180967 25408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:00.182171 25408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:00.192898 25419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:00.192873 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:00.192827 25420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:00.193616 25408 server_base.cc:1034] running on GCE node
I20260812 06:57:00.194319 25408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:00.195565 25408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:00.196801 25408 hybrid_clock.cc:648] HybridClock initialized: now 1786517820196742 us; error 28 us; skew 500 ppm
I20260812 06:57:00.197119 25408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:00.200212 25408 init.cc:377] Logged in from keytab as kudu/127.17.69.125@KRBTEST.COM (short username kudu)
I20260812 06:57:00.201517 25408 webserver.cc:460] Webserver started at http://127.17.69.125:43019/ using document root <none> and password file <none>
I20260812 06:57:00.202131 25408 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:00.202230 25408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:00.202549 25408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:00.205272 25408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data/instance:
uuid: "dee6d1f2b5644fec8c632caf4836e759"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "d86158604e89d9a783f2d289a3f9c4df"
server_key_iv: "222fbc30b8821560fd8c6b0e75cef232"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.206012 25408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal/instance:
uuid: "dee6d1f2b5644fec8c632caf4836e759"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "d86158604e89d9a783f2d289a3f9c4df"
server_key_iv: "222fbc30b8821560fd8c6b0e75cef232"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.210975 25408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260812 06:57:00.214149 25437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:00.215345 25408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:57:00.215497 25408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "dee6d1f2b5644fec8c632caf4836e759"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "d86158604e89d9a783f2d289a3f9c4df"
server_key_iv: "222fbc30b8821560fd8c6b0e75cef232"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.215689 25408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:00.248374 25408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:00.252067 25408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:00.252312 25408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:00.268980 25408 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.125:36151
I20260812 06:57:00.269011 25526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.125:36151 every 8 connection(s)
I20260812 06:57:00.270149 25408 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
I20260812 06:57:00.273561 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:00.279681 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25408
I20260812 06:57:00.279843 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-1/wal/instance
I20260812 06:57:00.280251 17685 external_mini_cluster.cc:1428] Setting key f24b724a64a3f38da9d8f8a389d3eef5
I20260812 06:57:00.279613 25528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:00.299381 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.281935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.17.69.126:38903)
0812 06:57:00.282138 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.282145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.283218 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 06:57:00.285379 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.285383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.285447 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.285529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.287310 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.287995 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.289926 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.290370 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.290467 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.294056 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.294061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.294064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.294095 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.296611 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.297272 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.297277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.297279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.297396 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.297774 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.297779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.297783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.298050 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.298190 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.298842 (+ 652us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.299059 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:57:00.300195 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.282305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38903 (local address 127.0.0.1:54604)
0812 06:57:00.283055 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.283100 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.285127 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.285769 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.285779 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.286441 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.287127 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.287150 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.288151 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.288156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.289780 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.289795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.290596 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.290603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.290890 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.291513 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.291539 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.293856 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.296740 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.296748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.296763 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.297162 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.297495 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.297500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.297504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.297670 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.298171 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.298178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.298500 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.298732 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.299173 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":278,"threads_started":1}
I20260812 06:57:00.301934 25528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.69.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.124@KRBTEST.COM" created.
I20260812 06:57:00.316360 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.302534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60694)
0812 06:57:00.302717 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.302736 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.302835 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.303840 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.303845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.304258 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.304424 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.304434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.305878 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.305883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.307051 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.307067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.307994 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.308000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.308190 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.308742 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.308765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.311094 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.314000 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.314005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.314010 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.314387 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.314724 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.314729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.314733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.314870 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.315448 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.315453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.315603 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.315711 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.315967 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:57:00.316479 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.302815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.17.69.125:36151)
0812 06:57:00.303446 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.303452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.303476 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:57:00.303514 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.303519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.303674 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.303780 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.304626 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.305726 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.307286 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.307835 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.307932 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.311290 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.311301 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.311315 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.311341 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.313886 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.314500 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.314506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.314508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.314624 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.314974 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.314978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.314980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.315327 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.315518 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.315900 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.316129 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":352,"threads_started":1}
I20260812 06:57:00.318717 25528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:00.319789 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.319311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56036)
0812 06:57:00.319452 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.319577 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:57:00.319998 25438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.321439 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.321124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56042)
0812 06:57:00.321262 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.321344 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 06:57:00.321646 25528 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
I20260812 06:57:00.349172 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:46735
--webserver_interface=127.17.69.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:34541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:00.370932 25528 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 1
I20260812 06:57:00.372140 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.371608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56048)
0812 06:57:00.371829 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.371954 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 06:57:00.373136 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.372729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56060)
0812 06:57:00.372904 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.372999 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 06:57:00.373334 25528 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.441565 25528 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 2
I20260812 06:57:00.442829 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.442258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56064)
0812 06:57:00.442514 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.442638 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 06:57:00.443854 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.443460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56076)
0812 06:57:00.443630 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.443714 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 06:57:00.444082 25528 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.471521 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } attempt: 5
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:57:00.484067 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.472314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.17.69.125:36151)
0812 06:57:00.472472 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.472477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.472497 (+ 20us) server_negotiation.cc:407] Connection header received
0812 06:57:00.472684 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.472689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.472776 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.472859 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.473647 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.474385 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.475515 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.475946 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.476055 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.479257 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.479262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.479265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.479296 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.481691 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.482285 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.482290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.482292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.482441 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.482779 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.482785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.482788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.483061 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.483191 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.483566 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.483778 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:57:00.484059 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.472207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60706)
0812 06:57:00.472407 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.472429 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.472556 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.472912 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.472917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.473315 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.473493 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.473503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.474512 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.474517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.475371 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.475383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.476087 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.476093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.476298 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.476937 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.476961 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.479079 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.481819 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.481824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.481829 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.482170 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.482542 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.482547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.482551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.482678 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.483174 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.483179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.483334 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.483451 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.483710 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 06:57:00.485306 25399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:00.486277 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.485896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56086)
0812 06:57:00.486048 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.486143 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 06:57:00.487190 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.486870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56094)
0812 06:57:00.487019 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.487096 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 06:57:00.487380 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.535315 25528 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 3
I20260812 06:57:00.536492 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.535933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56104)
0812 06:57:00.536172 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.536286 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:57:00.537583 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.537111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56110)
0812 06:57:00.537345 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.537445 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260812 06:57:00.537808 25528 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.538558 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 1
I20260812 06:57:00.539397 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.539007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56122)
0812 06:57:00.539179 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.539259 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:57:00.540385 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.540062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56136)
0812 06:57:00.540203 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.540288 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:57:00.540609 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.608794 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 2
I20260812 06:57:00.610000 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.609440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56152)
0812 06:57:00.609656 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.609779 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 06:57:00.611155 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.610755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56154)
0812 06:57:00.610913 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.611018 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 06:57:00.611438 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
W20260812 06:57:00.671496 25540 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:57:00.671798 25540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:00.671896 25540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:00.672009 25540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:00.678872 25540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:00.678952 25540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:00.679032 25540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:00.679101 25540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:00.679185 25540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:00.681037 25528 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 4
I20260812 06:57:00.682200 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.681657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56164)
0812 06:57:00.681900 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.682003 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 06:57:00.683161 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.682812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56180)
0812 06:57:00.682948 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.683030 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 06:57:00.683403 25528 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.689080 25540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:34541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:38903,127.17.69.125:36151,127.17.69.124:46735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:46735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.69.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:57:00.690135 25540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:00.691351 25540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:00.701854 25549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:00.701895 25550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:00.702003 25555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:00.702960 25540 server_base.cc:1034] running on GCE node
I20260812 06:57:00.703630 25540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:00.704824 25540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:00.706017 25540 hybrid_clock.cc:648] HybridClock initialized: now 1786517820705951 us; error 40 us; skew 500 ppm
I20260812 06:57:00.706305 25540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:00.711671 25540 init.cc:377] Logged in from keytab as kudu/127.17.69.124@KRBTEST.COM (short username kudu)
I20260812 06:57:00.712934 25540 webserver.cc:460] Webserver started at http://127.17.69.124:39829/ using document root <none> and password file <none>
I20260812 06:57:00.713533 25540 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:00.713629 25540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:00.713909 25540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:00.716815 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data/instance:
uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "8cf19e777c8db447c5a800fb063fc054"
server_key_iv: "a15ff3ba93e2f28d0817d61e285f0544"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.717553 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal/instance:
uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "8cf19e777c8db447c5a800fb063fc054"
server_key_iv: "a15ff3ba93e2f28d0817d61e285f0544"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.717658 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 3
I20260812 06:57:00.718691 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.718170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56188)
0812 06:57:00.718419 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.718514 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 06:57:00.719666 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.719274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56202)
0812 06:57:00.719420 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.719513 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:57:00.719916 25399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:46735: Network error: Client connection negotiation failed: client connection to 127.17.69.124:46735: connect: Connection refused (error 111)
I20260812 06:57:00.722863 25540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 06:57:00.725992 25572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:00.727170 25540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 06:57:00.727311 25540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0"
format_stamp: "Formatted at 2026-08-12 06:57:00 on dist-test-slave-6zbq"
server_key: "8cf19e777c8db447c5a800fb063fc054"
server_key_iv: "a15ff3ba93e2f28d0817d61e285f0544"
server_key_version: "encryptionkey@0"
I20260812 06:57:00.727474 25540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:00.744484 25540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:00.747889 25540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:00.748081 25540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:00.764277 25540 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.124:46735
I20260812 06:57:00.764295 25659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.124:46735 every 8 connection(s)
I20260812 06:57:00.765417 25540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
I20260812 06:57:00.768674 25660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:00.774386 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 25540
I20260812 06:57:00.774580 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786517466184829-17685-0/minicluster-data/master-2/wal/instance
I20260812 06:57:00.775134 17685 external_mini_cluster.cc:1428] Setting key a6dbb45d56a79e6def822ad12c15ea7e
I20260812 06:57:00.774811 25660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:57:00.794595 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.778105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.17.69.126:38903)
0812 06:57:00.778309 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.778315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.779451 (+ 1136us) server_negotiation.cc:407] Connection header received
0812 06:57:00.781227 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.781233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.781304 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.781389 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.782445 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.783148 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.784692 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.785111 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.785222 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.788507 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.788512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.788515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.788548 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.791481 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.792188 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.792194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.792198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.792360 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.792759 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.792763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.792766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.793065 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.793196 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.793976 (+ 780us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.794222 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 06:57:00.795469 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.777528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:38903 (local address 127.0.0.1:54610)
0812 06:57:00.779299 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.779345 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.780977 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.781471 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.781478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.781971 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.782281 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.782298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.783315 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.783319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.784555 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.784567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.785417 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.785426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.785648 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.786211 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.786232 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.788325 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.791607 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.791617 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.791624 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.792053 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.792465 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.792468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.792471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.792637 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.793215 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.793222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.793581 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.793861 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.794275 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1545,"mutex_wait_us":147,"thread_start_us":1412,"threads_started":1}
I20260812 06:57:00.795845 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.776870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.17.69.124:46735)
0812 06:57:00.778947 (+ 2077us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.778962 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.779528 (+ 566us) server_negotiation.cc:407] Connection header received
0812 06:57:00.780983 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.781006 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.781235 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.781400 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.782854 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.784054 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.785269 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.785850 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.785959 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.789777 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.789788 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.789802 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.789845 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.792779 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.793427 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.793433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.793436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.793563 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.793966 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.793971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.793974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.794430 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.794720 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.795239 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.795466 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":1851,"thread_start_us":1637,"threads_started":1}
I20260812 06:57:00.796957 25660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:00.810092 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.797720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.17.69.125:36151)
0812 06:57:00.797918 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.797922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.797938 (+ 16us) server_negotiation.cc:407] Connection header received
0812 06:57:00.797985 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.797988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.798043 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.798126 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.798896 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.799572 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.800759 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.801169 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.801259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.804220 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.804227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.804232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.804272 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.807063 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.807792 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.807797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.807800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.807989 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.808568 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.808574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.808578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.808901 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.809054 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.809574 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.809777 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 06:57:00.810809 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.797498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:36151 (local address 127.0.0.1:60714)
0812 06:57:00.797648 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.797665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.797762 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.798164 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.798167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.798569 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.798755 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.798767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.799721 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.799727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.800618 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.800633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.801300 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.801306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.801495 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.801960 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.801979 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.804020 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.807183 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.807187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.807189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.807644 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.808150 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.808154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.808158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.808337 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.809121 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.809126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.809301 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.810321 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.810537 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 06:57:00.811956 25660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:00.813616 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.798454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54618 (local address 127.17.69.126:38903)
0812 06:57:00.798580 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.798584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.798607 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:57:00.798733 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.798736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.798783 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.798868 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.799902 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.800563 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.801922 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.802333 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.802449 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.806689 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.806694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.806697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.806734 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.809812 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.810675 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.810681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.810685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.810842 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.811437 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.811443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.811446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.811792 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.812057 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.813127 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.813331 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:57:00.825891 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.812597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56234)
0812 06:57:00.812801 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.812817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.812914 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.813264 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.813268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.813703 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.813848 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.813859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.815115 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.815119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.816113 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.816123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.817285 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.817289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.817418 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.817913 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.817930 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.820387 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.823726 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.823731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.823736 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.824056 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.824363 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.824368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.824372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.824501 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.825011 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.825015 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.825162 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.825281 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.825529 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 06:57:00.825951 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.812733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.17.69.124:46735)
0812 06:57:00.813014 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.813020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.813050 (+ 30us) server_negotiation.cc:407] Connection header received
0812 06:57:00.813088 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.813092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.813149 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.813226 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.813983 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.814957 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.816258 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.817032 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.817156 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.820541 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.820547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.820551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.820580 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.823615 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.824155 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.824160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.824164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.824270 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.824592 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.824598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.824601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.824914 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.825043 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.825396 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.825652 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":79}
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:57:00.830874 25660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: Bootstrap starting.
I20260812 06:57:00.833272 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.819699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.17.69.125:36151)
0812 06:57:00.819838 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.819843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.819860 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:57:00.819905 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.819910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.819973 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.820066 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.821656 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.822653 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.823786 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.824196 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.824303 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.828439 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.828444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.828447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.828474 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.830918 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.831648 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.831651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.831654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.831776 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.832175 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.832178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.832180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.832411 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.832602 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.832892 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.833046 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 06:57:00.834424 25660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:00.835481 25660 log.cc:824] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:00.837579 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:57:00.838220 25660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: No bootstrap required, opened a new log
I20260812 06:57:00.840780 25353 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b7562b47719d47b2b036bc1e4f831401"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54618
I20260812 06:57:00.845508 25660 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.845782 25660 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbc8cffc5f441fc9cfea7ce8b245cd0, State: Initialized, Role: FOLLOWER
I20260812 06:57:00.846165 25660 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.847018 25667 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:00.847399 25667 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:00.847812 25660 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:57:00.856052 25681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:57:00.856208 25681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:57:00.853992 25540 master_runner.cc:186] Error getting master registration for 127.17.69.126:38903: OK, instance_id { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" instance_seqno: 1786517819766516 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:00.864522 25540 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:00.897125 25399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 4
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517819, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:57:00.910964 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.897725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56240)
0812 06:57:00.897895 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.897913 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.898003 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.898364 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.898369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.898756 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.898916 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.898926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.899924 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.899930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.900866 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.900878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.901565 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.901571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.901768 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.902406 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.902428 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.904984 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:00.908504 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.908509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.908513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.908971 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.909314 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:00.909319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:00.909323 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:00.909451 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:00.910030 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:00.910035 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:00.910185 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 06:57:00.910299 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 06:57:00.910576 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 06:57:00.910984 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.897807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.17.69.124:46735)
0812 06:57:00.897973 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.897978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.897999 (+ 21us) server_negotiation.cc:407] Connection header received
0812 06:57:00.898149 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.898155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.898215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.898293 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.899058 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.899781 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.901005 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.901429 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.901537 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.905172 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.905177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.905180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.905209 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:00.908384 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.909076 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.909082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.909086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.909211 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:00.909550 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:00.909556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:00.909560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:00.909901 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:00.910028 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:00.910437 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 06:57:00.910646 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 06:57:00.916682 25399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: Bootstrap starting.
I20260812 06:57:00.920650 25399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:00.921900 25399 log.cc:824] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:00.924950 25399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: No bootstrap required, opened a new log
I20260812 06:57:00.932166 25399 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.932550 25399 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7562b47719d47b2b036bc1e4f831401, State: Initialized, Role: FOLLOWER
I20260812 06:57:00.933000 25399 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.933943 25684 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:00.934288 25684 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:00.935078 25399 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:00.935163 25353 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:57:00.935365 25353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:00.937633 25353 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.938782 25353 leader_election.cc:288] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [CANDIDATE]: Term 1 election: Requested vote from peers dee6d1f2b5644fec8c632caf4836e759 (127.17.69.125:36151), bfbc8cffc5f441fc9cfea7ce8b245cd0 (127.17.69.124:46735)
I20260812 06:57:00.940261 25614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7562b47719d47b2b036bc1e4f831401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0"
I20260812 06:57:00.940734 25614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:00.944540 25614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7562b47719d47b2b036bc1e4f831401 in term 1.
I20260812 06:57:00.945312 25303 leader_election.cc:302] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7562b47719d47b2b036bc1e4f831401, bfbc8cffc5f441fc9cfea7ce8b245cd0; no voters:
I20260812 06:57:00.945744 25684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:00.957741 25684 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [term 1 LEADER]: Becoming Leader. State: Replica: b7562b47719d47b2b036bc1e4f831401, State: Running, Role: LEADER
I20260812 06:57:00.958724 25684 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:00.982617 25528 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } attempt: 5
I20260812 06:57:00.991286 25693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7562b47719d47b2b036bc1e4f831401. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:00.991668 25693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: This master's current role is: LEADER
Aug 12 06:57:00 dist-test-slave-6zbq krb5kdc[25253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
W20260812 06:57:01.001148 25582 tablet.cc:2369] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:01.002238 25714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:01.006227 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.968440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54622 (local address 127.17.69.126:38903)
0812 06:57:00.968605 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.968611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.978311 (+ 9700us) server_negotiation.cc:407] Connection header received
0812 06:57:00.979617 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.979623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.979701 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.979773 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.985976 (+ 6203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.987007 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.990288 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.990993 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.991090 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.997715 (+ 6625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:00.997722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:00.997728 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:00.997765 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:01.001533 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.003020 (+ 1487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.003025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.003029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.003234 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.004025 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.004030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.004033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.004400 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:01.004630 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.005608 (+ 978us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.005845 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 06:57:01.007426 25311 tablet.cc:2369] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:57:01.008487 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.983147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:46735 (local address 127.0.0.1:56250)
0812 06:57:00.983498 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:00.983516 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:00.983630 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:00.984531 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:00.984536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:00.985061 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:00.985273 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.985285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.986944 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.986950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.988304 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:00.988319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.989584 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.989600 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:00.989957 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:00.990627 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:00.990651 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:00.993299 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:01.005650 (+ 12351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:01.005656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:01.005661 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:01.006204 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:01.006703 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:01.006719 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:01.006723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:01.006888 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:01.007494 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:01.007500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:01.007659 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 06:57:01.007814 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 06:57:01.008102 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260812 06:57:01.009449 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.983923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.17.69.124:46735)
0812 06:57:00.984089 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.984094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.984112 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:57:00.984160 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.984164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.984232 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.984329 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.985484 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.986747 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.988511 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.989198 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.989317 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.000413 (+ 11096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:01.000424 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:01.000430 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:01.000471 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:01.005489 (+ 5018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.006396 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.006402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.006406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.006570 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.007014 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.007019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.007023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.007355 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:01.008749 (+ 1394us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.008889 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.009124 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 06:57:01.014959 25528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: Bootstrap starting.
I20260812 06:57:01.018247 25714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 06:57:01.018797 25528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:01.020845 25528 log.cc:824] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:01.028067 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.992228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.17.69.124:46735)
0812 06:57:01.004970 (+ 12742us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.004975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.006995 (+ 2020us) server_negotiation.cc:407] Connection header received
0812 06:57:01.009632 (+ 2637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.009641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.009709 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.009795 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.011447 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.012611 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.014306 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.015020 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.015126 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.018110 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:01.018117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:01.018122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:01.018161 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:01.021850 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.022679 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.022685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.022688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.022839 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.026253 (+ 3414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.026258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.026262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.026653 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:01.026880 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.027453 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.027667 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12203,"thread_start_us":3596,"threads_started":1}
I20260812 06:57:01.029381 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:00.978849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.17.69.125:36151)
0812 06:57:00.979023 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:00.979030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:00.990821 (+ 11791us) server_negotiation.cc:407] Connection header received
0812 06:57:00.991059 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:00.991065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:00.991135 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:00.991199 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:00.993443 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.994554 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.996575 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:00.997255 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:00.997345 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.022503 (+ 25158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:01.022510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:01.022515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:01.022554 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:01.026047 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.026902 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.026908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.026912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.027063 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:01.027493 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:01.027499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:01.027503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:01.027875 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:01.028110 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.028808 (+ 698us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.029046 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:57:01.039317 25528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: No bootstrap required, opened a new log
I20260812 06:57:01.063585 25614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [term 1 FOLLOWER]: Refusing update from remote peer b7562b47719d47b2b036bc1e4f831401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:01.064767 25684 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:01.076200 25528 raft_consensus.cc:348] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:01.076575 25528 raft_consensus.cc:729] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee6d1f2b5644fec8c632caf4836e759, State: Initialized, Role: FOLLOWER
I20260812 06:57:01.077080 25528 consensus_queue.cc:260] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } }
I20260812 06:57:01.066887 25278 master_runner.cc:186] Error getting master registration for 127.17.69.125:36151: OK, instance_id { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" instance_seqno: 1786517820264360 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:01.089092 25278 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:01.089959 25740 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.090787 25528 sys_catalog.cc:564] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:01.091619 25740 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:01.090833 25485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7562b47719d47b2b036bc1e4f831401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dee6d1f2b5644fec8c632caf4836e759"
I20260812 06:57:01.094267 25667 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7562b47719d47b2b036bc1e4f831401. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.094620 25667 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:01.091439 25484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:01.098975 25738 mvcc.cc:205] Tried to move back new op lower bound from 7317576995035815936 to 7317576994656702464. Current Snapshot: MvccSnapshot[applied={T|T < 7317576995035815936}]
I20260812 06:57:01.102402 25484 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [term 1 FOLLOWER]: Refusing update from remote peer b7562b47719d47b2b036bc1e4f831401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:01.103365 25693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:01.104888 25693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.105126 25693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:01.105964 25714 catalog_manager.cc:1347] Generated new cluster ID: 6895830f61f64177b9f2aa676f449d25
I20260812 06:57:01.106065 25714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:01.106495 25739 mvcc.cc:205] Tried to move back new op lower bound from 7317576995035815936 to 7317576994656702464. Current Snapshot: MvccSnapshot[applied={T|T < 7317576995035815936}]
I20260812 06:57:01.107590 25667 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.107820 25667 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:01.110540 25485 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7562b47719d47b2b036bc1e4f831401 in term 1.
I20260812 06:57:01.136674 25714 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:01.140369 25714 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:01.148072 25769 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.148526 25769 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:01.148861 25740 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7562b47719d47b2b036bc1e4f831401. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.149099 25740 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:01.155212 25693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7562b47719d47b2b036bc1e4f831401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7562b47719d47b2b036bc1e4f831401" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 38903 } } peers { permanent_uuid: "dee6d1f2b5644fec8c632caf4836e759" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 36151 } } peers { permanent_uuid: "bfbc8cffc5f441fc9cfea7ce8b245cd0" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 46735 } } }
I20260812 06:57:01.155514 25693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:01.159320 25714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7562b47719d47b2b036bc1e4f831401: Generated new TSK 0
I20260812 06:57:01.160161 25714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:01.210426 25772 catalog_manager.cc:1259] Loaded cluster ID: 6895830f61f64177b9f2aa676f449d25
I20260812 06:57:01.210610 25772 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: loading cluster ID for follower catalog manager: success
I20260812 06:57:01.214072 25772 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: acquiring CA information for follower catalog manager: success
I20260812 06:57:01.216519 25772 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dee6d1f2b5644fec8c632caf4836e759: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 06:57:01.327615 25534 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.317986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.17.69.125:36151)
0812 06:57:01.318195 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.318201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.324603 (+ 6402us) server_negotiation.cc:407] Connection header received
0812 06:57:01.326000 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.326006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.326127 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.326202 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.327397 (+ 1195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60748: BlockingRecv error: recv got EOF from 127.0.0.1:60748 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 06:57:01.327837 25403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.305061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.17.69.126:38903)
0812 06:57:01.305311 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.305318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.317145 (+ 11827us) server_negotiation.cc:407] Connection header received
0812 06:57:01.317356 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.317362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.317456 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.317585 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.327594 (+ 10009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54626: BlockingRecv error: recv got EOF from 127.0.0.1:54626 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 06:57:01.330399 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.325224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.17.69.124:46735)
0812 06:57:01.325406 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.325412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.328678 (+ 3266us) server_negotiation.cc:407] Connection header received
0812 06:57:01.328992 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.329001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.329110 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.329194 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.330130 (+ 936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56272: BlockingRecv error: recv got EOF from 127.0.0.1:56272 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 06:57:01.390261 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.372638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.17.69.125:36151)
0812 06:57:01.372795 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.372800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.382838 (+ 10038us) server_negotiation.cc:407] Connection header received
0812 06:57:01.383114 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.383120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.383328 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.383468 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.384170 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.385314 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.387315 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.387964 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.388060 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.388799 (+ 739us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.389486 (+ 687us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.389614 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.389838 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.389937 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 06:57:01.426044 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.423326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.17.69.124:46735)
0812 06:57:01.423701 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.423709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.424224 (+ 515us) server_negotiation.cc:407] Connection header received
0812 06:57:01.424543 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.424550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.424689 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.424766 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.425811 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56288: BlockingRecv error: recv got EOF from 127.0.0.1:56288 (error 108)
Metrics: {"server-negotiator.queue_time_us":221}
I20260812 06:57:01.437556 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.430896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.17.69.125:36151)
0812 06:57:01.431065 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.431072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.431584 (+ 512us) server_negotiation.cc:407] Connection header received
0812 06:57:01.431855 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.431861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.431982 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.432079 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.432502 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.433671 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.435521 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.436252 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.436372 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.436798 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.436958 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.437052 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.437196 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.437274 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 06:57:01.452461 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.444840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.17.69.124:46735)
0812 06:57:01.444995 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.445001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.450965 (+ 5964us) server_negotiation.cc:407] Connection header received
0812 06:57:01.451213 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.451219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.451313 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.451417 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.452245 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56290: BlockingRecv error: recv got EOF from 127.0.0.1:56290 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 06:57:01.468046 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.457947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.17.69.125:36151)
0812 06:57:01.458099 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.458108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.462000 (+ 3892us) server_negotiation.cc:407] Connection header received
0812 06:57:01.462172 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.462178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.462270 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.462401 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.462880 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.464061 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.465948 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.466637 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.466775 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.467230 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.467378 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.467478 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.467645 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.467729 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 06:57:01.480374 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.474944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.17.69.124:46735)
0812 06:57:01.475118 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.475123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.478897 (+ 3774us) server_negotiation.cc:407] Connection header received
0812 06:57:01.479145 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.479151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.479250 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.479350 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.480152 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56296: BlockingRecv error: recv got EOF from 127.0.0.1:56296 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 06:57:01.493551 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.487116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.17.69.125:36151)
0812 06:57:01.487289 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.487294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.487760 (+ 466us) server_negotiation.cc:407] Connection header received
0812 06:57:01.487918 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.487924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.488005 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.488098 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.488536 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.489669 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.491554 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.492251 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.492372 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.492839 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.492967 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.493065 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.493220 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.493297 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 06:57:01.501895 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.499838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.17.69.124:46735)
0812 06:57:01.500084 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.500089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.500578 (+ 489us) server_negotiation.cc:407] Connection header received
0812 06:57:01.500733 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.500739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.500819 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.500911 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.501693 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56304: BlockingRecv error: recv got EOF from 127.0.0.1:56304 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
I20260812 06:57:01.519958 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.513200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.17.69.125:36151)
0812 06:57:01.513379 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.513385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.513934 (+ 549us) server_negotiation.cc:407] Connection header received
0812 06:57:01.514102 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.514119 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.514199 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.514299 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.514757 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.515839 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.517661 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.518479 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.518603 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.519100 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.519256 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.519354 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.519561 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.519640 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 06:57:01.528215 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.526109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.17.69.124:46735)
0812 06:57:01.526272 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.526279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.526804 (+ 525us) server_negotiation.cc:407] Connection header received
0812 06:57:01.526980 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.526985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.527074 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.527189 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.528007 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56316: BlockingRecv error: recv got EOF from 127.0.0.1:56316 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 06:57:01.554292 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.547496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.17.69.125:36151)
0812 06:57:01.547697 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.547703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.548237 (+ 534us) server_negotiation.cc:407] Connection header received
0812 06:57:01.548427 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.548433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.548515 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.548622 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.549235 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.550454 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.552347 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.553000 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.553111 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.553586 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.553717 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.553796 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.553961 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.554041 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 06:57:01.570036 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.567973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.17.69.124:46735)
0812 06:57:01.568126 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.568131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.568616 (+ 485us) server_negotiation.cc:407] Connection header received
0812 06:57:01.568778 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.568784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.568875 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.568985 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.569837 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56320: BlockingRecv error: recv got EOF from 127.0.0.1:56320 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:57:01.610852 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.604960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.17.69.125:36151)
0812 06:57:01.605177 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.605183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.605534 (+ 351us) server_negotiation.cc:407] Connection header received
0812 06:57:01.605718 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.605722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.605802 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.605873 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.606408 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.607174 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.609130 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.609567 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.609680 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.610134 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.610230 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.610300 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.610480 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.610540 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 06:57:01.618039 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.616252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.17.69.124:46735)
0812 06:57:01.616429 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.616435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.616703 (+ 268us) server_negotiation.cc:407] Connection header received
0812 06:57:01.616830 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.616835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.616919 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.617016 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.617621 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56332: BlockingRecv error: recv got EOF from 127.0.0.1:56332 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:57:01.689694 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.684851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.17.69.125:36151)
0812 06:57:01.685046 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.685053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.685403 (+ 350us) server_negotiation.cc:407] Connection header received
0812 06:57:01.685586 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.685589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.685666 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.685743 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.686144 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.686884 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.688202 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.688576 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.688693 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.689058 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.689159 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.689233 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.689367 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.689446 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 06:57:01.696465 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.694764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.17.69.124:46735)
0812 06:57:01.694926 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.694931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.695257 (+ 326us) server_negotiation.cc:407] Connection header received
0812 06:57:01.695414 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.695418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.695488 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.695568 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.696267 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56348: BlockingRecv error: recv got EOF from 127.0.0.1:56348 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 06:57:01.832058 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:01.827507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.17.69.125:36151)
0812 06:57:01.827728 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.827734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.827959 (+ 225us) server_negotiation.cc:407] Connection header received
0812 06:57:01.828123 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.828127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.828199 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.828275 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:01.828644 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.829328 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.830636 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:01.831036 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:01.831137 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:01.831489 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:01.831586 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:01.831655 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:01.831741 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 06:57:01.831801 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 06:57:01.838985 25722 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:01.837032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.17.69.124:46735)
0812 06:57:01.837193 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:01.837198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:01.837598 (+ 400us) server_negotiation.cc:407] Connection header received
0812 06:57:01.837766 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:01.837770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:01.837852 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:01.837934 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:01.838767 (+ 833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56350: BlockingRecv error: recv got EOF from 127.0.0.1:56350 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 06:57:01.857980 25681 catalog_manager.cc:1259] Loaded cluster ID: 6895830f61f64177b9f2aa676f449d25
I20260812 06:57:01.858062 25681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: loading cluster ID for follower catalog manager: success
I20260812 06:57:01.860646 25681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: acquiring CA information for follower catalog manager: success
I20260812 06:57:01.862473 25681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfbc8cffc5f441fc9cfea7ce8b245cd0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:57:02.104274 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.099873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.17.69.125:36151)
0812 06:57:02.100110 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.100117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.100310 (+ 193us) server_negotiation.cc:407] Connection header received
0812 06:57:02.100471 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.100477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.100579 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.100667 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:02.100968 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.101701 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.102999 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.103400 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.103488 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.103789 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:02.103879 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:02.103940 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:02.104022 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 06:57:02.104081 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 06:57:02.114441 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.109467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.17.69.124:46735)
0812 06:57:02.109610 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.109614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.109896 (+ 282us) server_negotiation.cc:407] Connection header received
0812 06:57:02.110043 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.110046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.110193 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.110287 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:02.110638 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.111330 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.112501 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.112926 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.113014 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.113423 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:02.113965 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:02.114068 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:02.114144 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 06:57:02.114206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:57:02.127310 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.120555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60846 (local address 127.17.69.125:36151)
0812 06:57:02.120750 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.120756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.121635 (+ 879us) server_negotiation.cc:407] Connection header received
0812 06:57:02.121828 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.121834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.121915 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.121977 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:02.122758 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.123769 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.125634 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.126064 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.126152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.126624 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:02.126714 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:02.126787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:02.126955 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 06:57:02.127029 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 06:57:02.127344 25986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.119970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.17.69.126:38903)
0812 06:57:02.120458 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.120464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.120486 (+ 22us) server_negotiation.cc:407] Connection header received
0812 06:57:02.120719 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.120723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.120862 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.120993 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:02.121518 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.122538 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.124439 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.125035 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.125151 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.126375 (+ 1224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:02.126770 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:02.126869 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:02.126969 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 06:57:02.127050 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":200,"threads_started":1}
I20260812 06:57:02.127467 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:02.121861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.17.69.124:46735)
0812 06:57:02.122005 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:02.122010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:02.122121 (+ 111us) server_negotiation.cc:407] Connection header received
0812 06:57:02.122380 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:02.122384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:02.122454 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:02.122535 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:02.123098 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.123749 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.125719 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:02.126094 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:02.126220 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:02.126768 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:02.126891 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:02.126948 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:02.127164 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 06:57:02.127235 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:57:02.131742 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25278
I20260812 06:57:02.132766 25391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:02.238571 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25278
I20260812 06:57:02.264660 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25408
I20260812 06:57:02.265630 25518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:02.368381 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25408
I20260812 06:57:02.392364 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 25540
I20260812 06:57:02.393429 25654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:02.497844 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 25540
2026-08-12T06:57:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4703 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 06:57:02 dist-test-slave-6zbq krb5kdc[25998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 06:57:02 dist-test-slave-6zbq krb5kdc[25998](info): set up 2 sockets
Aug 12 06:57:02 dist-test-slave-6zbq krb5kdc[25998](info): commencing operation
krb5kdc: starting...
W20260812 06:57:03.760694 17685 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/test-user.keytab.
Aug 12 06:57:03 dist-test-slave-6zbq krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T06:57:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:57:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.69.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
Entry for principal kudu/127.17.69.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab.
I20260812 06:57:04.039407 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37733
--webserver_interface=127.17.69.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 06:57:04.359673 26025 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:57:04.359967 26025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:04.360059 26025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:04.360172 26025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 06:57:04.367003 26025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:04.367085 26025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:04.367165 26025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:04.367228 26025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:04.367311 26025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:04.376591 26025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.126.keytab
--principal=kudu/127.17.69.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.126:37733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.69.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:57:04.377703 26025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:04.378942 26025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:04.389586 26034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:04.389693 26035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:04.389730 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:04.390640 26025 server_base.cc:1034] running on GCE node
I20260812 06:57:04.391355 26025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:04.392560 26025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:04.393769 26025 hybrid_clock.cc:648] HybridClock initialized: now 1786517824393709 us; error 31 us; skew 500 ppm
I20260812 06:57:04.394049 26025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:04 dist-test-slave-6zbq krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:04.396920 26025 init.cc:377] Logged in from keytab as kudu/127.17.69.126@KRBTEST.COM (short username kudu)
I20260812 06:57:04.398222 26025 webserver.cc:460] Webserver started at http://127.17.69.126:46261/ using document root <none> and password file <none>
I20260812 06:57:04.398849 26025 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:04.398952 26025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:04.399242 26025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:04.401973 26025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data/instance:
uuid: "ab36b3105cc5497191fe07f9fbdb87d5"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "4f134f0fd4ee05c8b7d54adc86b5c441"
server_key_iv: "238c2283ab608dfbbcaf1bbe9a724fd0"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.402745 26025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance:
uuid: "ab36b3105cc5497191fe07f9fbdb87d5"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "4f134f0fd4ee05c8b7d54adc86b5c441"
server_key_iv: "238c2283ab608dfbbcaf1bbe9a724fd0"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.407408 26025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 06:57:04.410547 26048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:04.411701 26025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:57:04.411845 26025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal
uuid: "ab36b3105cc5497191fe07f9fbdb87d5"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "4f134f0fd4ee05c8b7d54adc86b5c441"
server_key_iv: "238c2283ab608dfbbcaf1bbe9a724fd0"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.412025 26025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:04.426009 26025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:04.429802 26025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:04.430028 26025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:04.446894 26025 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.126:37733
I20260812 06:57:04.446924 26137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.126:37733 every 8 connection(s)
I20260812 06:57:04.448076 26025 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/data/info.pb
I20260812 06:57:04.451478 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:04.454799 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26025
I20260812 06:57:04.454981 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-0/wal/instance
I20260812 06:57:04.455544 17685 external_mini_cluster.cc:1428] Setting key 65396525fec42fe29dff60f6ac9fee6b
I20260812 06:57:04.458942 26138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:04 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:04.480571 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.463177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.17.69.126:37733)
0812 06:57:04.463822 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.463830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.463875 (+ 45us) server_negotiation.cc:407] Connection header received
0812 06:57:04.465194 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.465201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.465471 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.465668 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.466938 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.468508 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.470142 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.470905 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.471064 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.474180 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:04.474188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:04.474197 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:04.474231 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:04.476758 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.477373 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.477380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.477382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.477519 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.477874 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.477878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.477880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.478303 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:04.478518 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:04.479004 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 06:57:04.479256 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":445,"thread_start_us":295,"threads_started":1}
I20260812 06:57:04.480590 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.461996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37733 (local address 127.0.0.1:40724)
0812 06:57:04.462771 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.462820 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.464820 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.465694 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.465700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.466299 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.466728 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.466747 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.468657 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.468661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.470003 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.470016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.471120 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.471127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.471369 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.471944 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.471969 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.473976 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:04.476872 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.476883 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.476894 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.477260 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.477620 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.477625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.477629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.477778 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.478453 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:04.478460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:04.478658 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 06:57:04.478865 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 06:57:04.479386 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":179,"thread_start_us":339,"threads_started":1}
I20260812 06:57:04.483979 26138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:04.484921 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.484562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56370)
0812 06:57:04.484703 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.484804 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:57:04.485139 26051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
I20260812 06:57:04.486589 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.486224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56382)
0812 06:57:04.486392 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.486456 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 06:57:04.486784 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:41447: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.69.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
Entry for principal kudu/127.17.69.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab.
I20260812 06:57:04.523012 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:41447
--webserver_interface=127.17.69.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:04.536053 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } attempt: 1
I20260812 06:57:04.537078 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.536645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56392)
0812 06:57:04.536839 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.536956 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:57:04.538053 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.537720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56398)
0812 06:57:04.537861 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.537924 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 06:57:04.538216 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:41447: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
I20260812 06:57:04.606431 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } attempt: 2
I20260812 06:57:04.607583 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.607101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56406)
0812 06:57:04.607301 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.607396 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 06:57:04.608831 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.608455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56416)
0812 06:57:04.608611 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.608697 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 06:57:04.609006 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:41447: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
I20260812 06:57:04.700253 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } attempt: 3
I20260812 06:57:04.701385 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.700949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56420)
0812 06:57:04.701161 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.701254 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 06:57:04.702517 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.702128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56426)
0812 06:57:04.702274 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.702376 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 06:57:04.702697 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:41447: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
W20260812 06:57:04.843827 26148 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:57:04.844112 26148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:04.844211 26148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:04.844328 26148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 06:57:04.845904 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } attempt: 4
I20260812 06:57:04.847070 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.846566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56440)
0812 06:57:04.846855 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.846940 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 06:57:04.848060 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.847692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56444)
0812 06:57:04.847809 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.847916 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 06:57:04.848292 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.125:41447: Network error: Client connection negotiation failed: client connection to 127.17.69.125:41447: connect: Connection refused (error 111)
W20260812 06:57:04.851881 26148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:04.851965 26148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:04.852053 26148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:04.852114 26148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:57:04.852195 26148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:04.861339 26148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.125.keytab
--principal=kudu/127.17.69.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.125:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.69.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:57:04.862449 26148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:04.863626 26148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:04.874439 26166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:04.874482 26164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:04.874503 26176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:04.875626 26148 server_base.cc:1034] running on GCE node
I20260812 06:57:04.876298 26148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:04.877475 26148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:04.878682 26148 hybrid_clock.cc:648] HybridClock initialized: now 1786517824878616 us; error 37 us; skew 500 ppm
I20260812 06:57:04.878973 26148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:04 dist-test-slave-6zbq krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:04.881861 26148 init.cc:377] Logged in from keytab as kudu/127.17.69.125@KRBTEST.COM (short username kudu)
I20260812 06:57:04.883165 26148 webserver.cc:460] Webserver started at http://127.17.69.125:38749/ using document root <none> and password file <none>
I20260812 06:57:04.883778 26148 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:04.883879 26148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:04.884161 26148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:04.886906 26148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data/instance:
uuid: "63a90d56f8c7434d9e740faf8dde9998"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "93b5e65fc589e6ba46754a8770d0a553"
server_key_iv: "37d476bde3b7b2df4620f9b77253f25a"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.887645 26148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal/instance:
uuid: "63a90d56f8c7434d9e740faf8dde9998"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "93b5e65fc589e6ba46754a8770d0a553"
server_key_iv: "37d476bde3b7b2df4620f9b77253f25a"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.892311 26148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 06:57:04.895423 26186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:04.896589 26148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 06:57:04.896740 26148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal
uuid: "63a90d56f8c7434d9e740faf8dde9998"
format_stamp: "Formatted at 2026-08-12 06:57:04 on dist-test-slave-6zbq"
server_key: "93b5e65fc589e6ba46754a8770d0a553"
server_key_iv: "37d476bde3b7b2df4620f9b77253f25a"
server_key_version: "encryptionkey@0"
I20260812 06:57:04.896924 26148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:04.920291 26148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:04.924216 26148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:04.924458 26148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:04.941401 26148 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.125:41447
I20260812 06:57:04.941421 26281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.125:41447 every 8 connection(s)
I20260812 06:57:04.942647 26148 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/data/info.pb
I20260812 06:57:04.945964 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:04.950927 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26148
I20260812 06:57:04.951109 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-1/wal/instance
I20260812 06:57:04.951615 17685 external_mini_cluster.cc:1428] Setting key b99fcc75efa3cc906c5f60ad5afa8f79
I20260812 06:57:04.952631 26286 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:04 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:04.971902 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.955602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.17.69.126:37733)
0812 06:57:04.955806 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.955812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.956848 (+ 1036us) server_negotiation.cc:407] Connection header received
0812 06:57:04.958673 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.958679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.958763 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.958871 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.960209 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.961115 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.962558 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.963101 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.963216 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.966425 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:04.966431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:04.966435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:04.966472 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:04.968945 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.969796 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.969800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.969802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.969919 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.970316 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.970319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.970321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.970615 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:04.970727 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:04.971516 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 06:57:04.971681 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 06:57:04.973204 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.955928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37733 (local address 127.0.0.1:40732)
0812 06:57:04.956676 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.956719 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.958425 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.959048 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.959055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.959534 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.960048 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.960065 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.961248 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.961252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.962401 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.962415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.963278 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.963283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.963483 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.964064 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.964085 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.966192 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:04.969081 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.969089 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.969102 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.969693 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.970012 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.970016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.970019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.970224 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.970739 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:04.970747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:04.971185 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 06:57:04.971463 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 06:57:04.971889 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":310,"threads_started":1}
I20260812 06:57:04.974800 26286 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:04 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.69.124@KRBTEST.COM; defaulting to no policy
I20260812 06:57:04.988679 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.975376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56448)
0812 06:57:04.975541 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.975565 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:04.975666 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:04.976779 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:04.976782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:04.977118 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:04.977260 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.977268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.979036 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.979040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.979850 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:04.979860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.980791 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.980795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:04.980927 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.981466 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:04.981481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:04.983674 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:04.986691 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.986695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.986698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.987033 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.987400 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:04.987403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:04.987406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:04.987537 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:04.988057 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:04.988061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:04.988172 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 06:57:04.988273 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 06:57:04.988445 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.69.124@KRBTEST.COM" created.
I20260812 06:57:04.989043 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.975679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.17.69.125:41447)
0812 06:57:04.976334 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:04.976341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:04.976370 (+ 29us) server_negotiation.cc:407] Connection header received
0812 06:57:04.976417 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:04.976423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:04.976618 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:04.976764 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:04.977424 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.978914 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.979980 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:04.980666 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:04.980791 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:04.983868 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:04.983878 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:04.983889 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:04.983921 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:04.986575 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.987137 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.987142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.987145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.987309 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:04.987626 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:04.987629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:04.987631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:04.987955 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:04.988111 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:04.988472 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 06:57:04.988654 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":488,"thread_start_us":345,"threads_started":1}
I20260812 06:57:04.991199 26286 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:04.992336 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.991863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34338)
0812 06:57:04.992030 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.992177 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 06:57:04.992620 26190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:04.994026 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:04.993687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34344)
0812 06:57:04.993823 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:04.993909 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 06:57:04.994277 26286 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
Entry for principal kudu/127.17.69.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab.
I20260812 06:57:05.022413 17685 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
/tmp/dist-test-tasknIa6XT/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:39553
--webserver_interface=127.17.69.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.69.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 06:57:05.043598 26286 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 1
I20260812 06:57:05.044723 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.044289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34356)
0812 06:57:05.044476 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.044578 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 06:57:05.045645 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.045312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34366)
0812 06:57:05.045467 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.045540 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 06:57:05.045837 26286 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.114045 26286 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 2
I20260812 06:57:05.115125 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.114655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34370)
0812 06:57:05.114887 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.114974 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 06:57:05.116008 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.115686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34374)
0812 06:57:05.115830 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.115894 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 06:57:05.116247 26286 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.147516 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } attempt: 5
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:57:05.160053 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.148198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56460)
0812 06:57:05.148402 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.148422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.148564 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.148925 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.148930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.149331 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.149512 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.149522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.150533 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.150537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.151396 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.151409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.152134 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.152140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.152341 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.152913 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.152932 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.154925 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.157770 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.157776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.157780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.158134 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.158519 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.158524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.158527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.158658 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.159140 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.159144 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.159297 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.159436 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.159702 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 06:57:05.160053 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.148245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.17.69.125:41447)
0812 06:57:05.148424 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.148430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.148501 (+ 71us) server_negotiation.cc:407] Connection header received
0812 06:57:05.148712 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.148716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.148780 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.148868 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.149675 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.150396 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.151548 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.151995 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.152105 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.155106 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.155112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.155115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.155148 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.157643 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.158251 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.158256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.158258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.158410 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.158759 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.158764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.158768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.159026 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.159157 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.159559 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.159772 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 06:57:05.161491 26138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:05.162549 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.162146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34376)
0812 06:57:05.162313 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.162420 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:57:05.163394 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.163088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34382)
0812 06:57:05.163220 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.163296 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 06:57:05.163573 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.207430 26286 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 3
I20260812 06:57:05.208458 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.208041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34392)
0812 06:57:05.208228 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.208313 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 06:57:05.209422 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.209028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34394)
0812 06:57:05.209236 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.209310 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260812 06:57:05.209619 26286 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.214757 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 1
I20260812 06:57:05.215579 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.215200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34396)
0812 06:57:05.215363 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.215437 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 06:57:05.216427 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.216127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34402)
0812 06:57:05.216262 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.216328 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 06:57:05.216609 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.284817 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 2
I20260812 06:57:05.285856 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.285395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34416)
0812 06:57:05.285613 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.285701 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 06:57:05.286772 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.286453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34426)
0812 06:57:05.286588 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.286679 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 06:57:05.286963 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
W20260812 06:57:05.346058 26298 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:57:05.346416 26298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:57:05.346516 26298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:57:05.346632 26298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 06:57:05.352821 26286 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 4
W20260812 06:57:05.353597 26298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:57:05.353674 26298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 06:57:05.353768 26298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:57:05.353833 26298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260812 06:57:05.353897 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.353374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34442)
0812 06:57:05.353600 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.353703 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 06:57:05.353915 26298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:57:05.354908 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.354609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34446)
0812 06:57:05.354744 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.354813 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 06:57:05.355105 26286 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.363679 26298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.69.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.69.126:37733,127.17.69.125:41447,127.17.69.124:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/krb5kdc/kudu_127.17.69.124.keytab
--principal=kudu/127.17.69.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.69.124:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.69.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 05:55:59 UTC on ccycloud.quasar-tbxajw.root.comops.site
build id 4581995
ASAN enabled
I20260812 06:57:05.364832 26298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:57:05.366057 26298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 06:57:05.376791 26307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:05.376828 26309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:57:05.376860 26313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:57:05.377988 26298 server_base.cc:1034] running on GCE node
I20260812 06:57:05.378739 26298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:57:05.379925 26298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:57:05.381134 26298 hybrid_clock.cc:648] HybridClock initialized: now 1786517825381069 us; error 37 us; skew 500 ppm
I20260812 06:57:05.381433 26298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517825, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 06:57:05.384423 26298 init.cc:377] Logged in from keytab as kudu/127.17.69.124@KRBTEST.COM (short username kudu)
I20260812 06:57:05.385856 26298 webserver.cc:460] Webserver started at http://127.17.69.124:38723/ using document root <none> and password file <none>
I20260812 06:57:05.386561 26298 fs_manager.cc:359] Metadata directory not provided
I20260812 06:57:05.386649 26298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:57:05.386904 26298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:57:05.389634 26298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data/instance:
uuid: "0473628c45d04868a28980b28609078a"
format_stamp: "Formatted at 2026-08-12 06:57:05 on dist-test-slave-6zbq"
server_key: "ccfc03ab217be0f37bdd2a386ba32523"
server_key_iv: "f3dbe0dcc20bccef9acfcd623853fdbf"
server_key_version: "encryptionkey@0"
I20260812 06:57:05.390364 26298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal/instance:
uuid: "0473628c45d04868a28980b28609078a"
format_stamp: "Formatted at 2026-08-12 06:57:05 on dist-test-slave-6zbq"
server_key: "ccfc03ab217be0f37bdd2a386ba32523"
server_key_iv: "f3dbe0dcc20bccef9acfcd623853fdbf"
server_key_version: "encryptionkey@0"
I20260812 06:57:05.393139 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 3
I20260812 06:57:05.394091 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.393660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34460)
0812 06:57:05.393852 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.393933 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 06:57:05.395090 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.394704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34474)
0812 06:57:05.394915 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.394988 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 06:57:05.395237 26298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
W20260812 06:57:05.395315 26138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.69.124:39553: Network error: Client connection negotiation failed: client connection to 127.17.69.124:39553: connect: Connection refused (error 111)
I20260812 06:57:05.398427 26331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 06:57:05.399572 26298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 06:57:05.399714 26298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data,/tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal
uuid: "0473628c45d04868a28980b28609078a"
format_stamp: "Formatted at 2026-08-12 06:57:05 on dist-test-slave-6zbq"
server_key: "ccfc03ab217be0f37bdd2a386ba32523"
server_key_iv: "f3dbe0dcc20bccef9acfcd623853fdbf"
server_key_version: "encryptionkey@0"
I20260812 06:57:05.399889 26298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:57:05.423273 26298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 06:57:05.426792 26298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:57:05.427019 26298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:57:05.443547 26298 rpc_server.cc:307] RPC server started. Bound to: 127.17.69.124:39553
I20260812 06:57:05.443584 26419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.69.124:39553 every 8 connection(s)
I20260812 06:57:05.444712 26298 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/data/info.pb
I20260812 06:57:05.448073 26421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:57:05.450729 17685 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu as pid 26298
I20260812 06:57:05.450909 17685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknIa6XT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786517466184829-17685-0/minicluster-data/master-2/wal/instance
I20260812 06:57:05.451433 17685 external_mini_cluster.cc:1428] Setting key e6d629810b51cad951f7001241890f09
I20260812 06:57:05.455997 26421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517825, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:05.473095 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.458954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.17.69.126:37733)
0812 06:57:05.459105 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.459109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.459697 (+ 588us) server_negotiation.cc:407] Connection header received
0812 06:57:05.459871 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.459876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.459935 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.460014 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.461225 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.461896 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.463510 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.463937 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.464046 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.467407 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.467412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.467415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.467446 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.470128 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.470866 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.470869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.470871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.471040 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.471396 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.471401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.471404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.471759 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.471895 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.472513 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.472744 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 06:57:05.473582 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.458955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.126:37733 (local address 127.0.0.1:40748)
0812 06:57:05.459576 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.459608 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.459757 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.460103 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.460111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.460739 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.461056 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.461069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.462052 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.462056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.463371 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.463384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.464148 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.464155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.464305 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.464872 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.464892 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.467229 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.470252 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.470262 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.470270 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.470757 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.471141 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.471144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.471146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.471284 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.471887 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.471894 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.472145 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.472363 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.472637 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"spinlock_wait_cycles":15744,"thread_start_us":315,"threads_started":1}
I20260812 06:57:05.473614 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.453209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.17.69.124:39553)
0812 06:57:05.455214 (+ 2005us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.455226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.455261 (+ 35us) server_negotiation.cc:407] Connection header received
0812 06:57:05.456776 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.456797 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.457153 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.457471 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.458922 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.460545 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.461945 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.462667 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.462786 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.467067 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.467079 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.467092 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.467129 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.469655 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.470279 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.470288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.470291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.470545 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.470946 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.470951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.470954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.471547 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.471845 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.472140 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.472474 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1819,"thread_start_us":1637,"threads_started":1}
I20260812 06:57:05.475055 26421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517825, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.126@KRBTEST.COM
I20260812 06:57:05.488225 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.475604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.125:41447 (local address 127.0.0.1:56476)
0812 06:57:05.475802 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.475815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.475895 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.476334 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.476338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.476722 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.476894 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.476905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.478004 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.478008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.479076 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.479090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.479750 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.479756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.479953 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.480453 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.480468 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.482803 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.485845 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.485850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.485854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.486308 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.486729 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.486735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.486738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.486904 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.487385 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.487390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.487556 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.487716 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.487955 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260812 06:57:05.488300 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.475834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.17.69.125:41447)
0812 06:57:05.476014 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.476020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.476038 (+ 18us) server_negotiation.cc:407] Connection header received
0812 06:57:05.476098 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.476103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.476180 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.476294 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.477061 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.477858 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.479228 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.479617 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.479730 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.482981 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.482986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.482989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.483021 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.485720 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.486441 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.486446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.486449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.486620 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.487003 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.487006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.487008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.487265 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.487375 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.487793 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.487941 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 06:57:05.489141 26421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:57:05.490851 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.476150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.17.69.126:37733)
0812 06:57:05.476301 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.476306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.476321 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:57:05.476459 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.476463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.476512 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.476578 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.477787 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.478607 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.479983 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.480521 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.480613 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.484999 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.485004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.485007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.485035 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.487922 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.488616 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.488622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.488625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.488812 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.489203 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.489207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.489210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.489538 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.489790 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.490485 (+ 695us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.490643 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517825, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.124@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.69.125@KRBTEST.COM
I20260812 06:57:05.502064 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.489808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34496)
0812 06:57:05.489942 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.489959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.490053 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.490313 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.490317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.490672 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.490790 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.490797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.491799 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.491804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.492858 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.492870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.493490 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.493496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.493696 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.494153 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.494173 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.496266 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.499712 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.499716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.499718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.500208 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.500632 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.500637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.500640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.500798 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.501299 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.501303 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.501497 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.501628 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.501821 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 06:57:05.502319 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.489824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.17.69.124:39553)
0812 06:57:05.489952 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.489957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.490003 (+ 46us) server_negotiation.cc:407] Connection header received
0812 06:57:05.490145 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.490150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.490212 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.490284 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.490911 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.491702 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.492975 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.493354 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.493459 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.496434 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.496439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.496442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.496469 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.499575 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.500322 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.500326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.500328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.500463 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.500903 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.500906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.500908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.501169 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.501281 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.501748 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.501977 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 06:57:05.506819 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.492307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.17.69.125:41447)
0812 06:57:05.492457 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.492462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.492479 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:57:05.492524 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.492529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.492588 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.492681 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.493674 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.494658 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.495825 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.496454 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.496538 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.501506 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.501513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.501518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.501551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.504456 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.505061 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.505067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.505070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.505200 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.505542 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.505547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.505550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.505811 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.506058 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.506310 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.506538 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:57:05.507069 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: Bootstrap starting.
I20260812 06:57:05.510609 26421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:05.511125 17685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 06:57:05.511890 26421 log.cc:824] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:05.514329 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: No bootstrap required, opened a new log
I20260812 06:57:05.513352 26087 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ab36b3105cc5497191fe07f9fbdb87d5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40754
I20260812 06:57:05.521363 26421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.521689 26421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0473628c45d04868a28980b28609078a, State: Initialized, Role: FOLLOWER
I20260812 06:57:05.522104 26421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.522935 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.523233 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.523973 26421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:57:05.532616 26454 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:57:05.532753 26454 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:57:05.530777 26298 master_runner.cc:186] Error getting master registration for 127.17.69.126:37733: OK, instance_id { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" instance_seqno: 1786517824442068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:05.541460 26298 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:05.572544 26138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 4
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.126@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:57:05.587872 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.573224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34508)
0812 06:57:05.573404 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.573424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.573525 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.574041 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.574047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.574609 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.574836 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.574851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.575883 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.575889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.577354 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.577372 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.578414 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.578421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.578639 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.579315 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.579339 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.582133 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.585151 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.585157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.585161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.585511 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.585883 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.585889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.585893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.586078 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.586873 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.586879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.587063 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.587219 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.587488 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 06:57:05.587944 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.573324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.17.69.124:39553)
0812 06:57:05.573529 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.573535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.573558 (+ 23us) server_negotiation.cc:407] Connection header received
0812 06:57:05.573655 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.573660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.573739 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.573848 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.574978 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.575725 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.577510 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.578234 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.578374 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.582330 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.582335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.582364 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.582403 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.585019 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.585626 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.585631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.585633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.585756 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.586240 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.586246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.586250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.586731 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.586878 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.587347 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.587579 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 06:57:05.593731 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: Bootstrap starting.
I20260812 06:57:05.597664 26138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:05.598978 26138 log.cc:824] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:05.602173 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: No bootstrap required, opened a new log
I20260812 06:57:05.609560 26138 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.609979 26138 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab36b3105cc5497191fe07f9fbdb87d5, State: Initialized, Role: FOLLOWER
I20260812 06:57:05.610484 26138 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.611450 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.611824 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.612581 26138 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:05.612682 26087 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:57:05.612861 26087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:05.615461 26087 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.617434 26378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0473628c45d04868a28980b28609078a"
I20260812 06:57:05.617915 26378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:05.621361 26378 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab36b3105cc5497191fe07f9fbdb87d5 in term 1.
I20260812 06:57:05.621948 26050 leader_election.cc:302] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0473628c45d04868a28980b28609078a, ab36b3105cc5497191fe07f9fbdb87d5; no voters:
I20260812 06:57:05.622542 26087 leader_election.cc:288] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [CANDIDATE]: Term 1 election: Requested vote from peers 63a90d56f8c7434d9e740faf8dde9998 (127.17.69.125:41447), 0473628c45d04868a28980b28609078a (127.17.69.124:39553)
I20260812 06:57:05.622478 26459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:57:05.623385 26459 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [term 1 LEADER]: Becoming Leader. State: Replica: ab36b3105cc5497191fe07f9fbdb87d5, State: Running, Role: LEADER
I20260812 06:57:05.623878 26459 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.627213 26475 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab36b3105cc5497191fe07f9fbdb87d5. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.627610 26475 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:05.641613 26479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 06:57:05.654289 26286 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } attempt: 5
I20260812 06:57:05.659299 26479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 06:57:05.678812 26340 tablet.cc:2369] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:57:05.683888 26057 tablet.cc:2369] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 06:57:05 dist-test-slave-6zbq krb5kdc[25998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786517824, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.69.125@KRBTEST.COM for kudu/127.17.69.124@KRBTEST.COM
I20260812 06:57:05.688956 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.651798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.17.69.124:39553)
0812 06:57:05.651971 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.651977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.668544 (+ 16567us) server_negotiation.cc:407] Connection header received
0812 06:57:05.669004 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.669010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.669093 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.669164 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.671199 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.672373 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.675549 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.676245 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.676347 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.681527 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.681538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.681543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.681582 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.685283 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.686493 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.686499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.686502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.686669 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.687127 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.687133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.687136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.687493 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.687751 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.688275 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.688594 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 06:57:05.679368 26025 master_runner.cc:186] Error getting master registration for 127.17.69.125:41447: OK, instance_id { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" instance_seqno: 1786517824936547 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 06:57:05.690915 26025 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 06:57:05.696519 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.654848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.69.124:39553 (local address 127.0.0.1:34536)
0812 06:57:05.655089 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 06:57:05.655108 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 06:57:05.655231 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 06:57:05.675196 (+ 19965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 06:57:05.675203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 06:57:05.675698 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 06:57:05.675903 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.675916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.682102 (+ 6186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.682108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.683558 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 06:57:05.683574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.685555 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.685562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 06:57:05.685807 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.686370 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 06:57:05.686394 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 06:57:05.689059 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 06:57:05.693719 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.693725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.693729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.694253 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.694727 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 06:57:05.694733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 06:57:05.694737 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 06:57:05.694930 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 06:57:05.695553 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 06:57:05.695559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 06:57:05.695722 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 06:57:05.695855 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 06:57:05.696136 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260812 06:57:05.697610 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.655498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.17.69.124:39553)
0812 06:57:05.674763 (+ 19265us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.674769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.674793 (+ 24us) server_negotiation.cc:407] Connection header received
0812 06:57:05.674839 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.674844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.674915 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.680428 (+ 5513us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.680718 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.681928 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.684625 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.685375 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.686086 (+ 711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.689311 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.689318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.689323 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.689358 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.693588 (+ 4230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.694417 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.694422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.694426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.694575 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.695074 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.695078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.695082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.695416 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.696876 (+ 1460us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.696994 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.697208 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19127,"thread_start_us":19495,"threads_started":1}
I20260812 06:57:05.703186 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: Bootstrap starting.
I20260812 06:57:05.706198 26378 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [term 1 FOLLOWER]: Refusing update from remote peer ab36b3105cc5497191fe07f9fbdb87d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:05.707135 26475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:05.713742 26286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: Neither blocks nor log segments found. Creating new log.
I20260812 06:57:05.715512 26286 log.cc:824] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: Log is configured to *not* fsync() on all Append() calls
I20260812 06:57:05.716886 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.649938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.17.69.126:37733)
0812 06:57:05.650118 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.650124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.662954 (+ 12830us) server_negotiation.cc:407] Connection header received
0812 06:57:05.664132 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.664139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.664223 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.664297 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.674408 (+ 10111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.675523 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.695221 (+ 19698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.695888 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.696015 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.705859 (+ 9844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.705866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.705871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.705912 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.709778 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.711349 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.711359 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.711362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.711707 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.712619 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.712625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.712629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.713568 (+ 939us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.714184 (+ 616us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.715427 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.716201 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 06:57:05.729182 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:05.661467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.17.69.125:41447)
0812 06:57:05.661644 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.661650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.680507 (+ 18857us) server_negotiation.cc:407] Connection header received
0812 06:57:05.694573 (+ 14066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.694578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.694647 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.694725 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.696991 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.698032 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.705892 (+ 7860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:05.706558 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:05.706931 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:05.722168 (+ 15237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 06:57:05.722176 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 06:57:05.722180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 06:57:05.722228 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 06:57:05.725700 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.726489 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.726494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.726497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.726656 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 06:57:05.727054 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 06:57:05.727059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 06:57:05.727065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 06:57:05.727533 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 06:57:05.727776 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:05.728192 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 06:57:05.728593 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 06:57:05.732702 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: No bootstrap required, opened a new log
I20260812 06:57:05.735567 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab36b3105cc5497191fe07f9fbdb87d5. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.735884 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.736156 26501 mvcc.cc:205] Tried to move back new op lower bound from 7317577014079565824 to 7317577013759864832. Current Snapshot: MvccSnapshot[applied={T|T < 7317577014079565824}]
I20260812 06:57:05.742982 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.743216 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.752460 26479 catalog_manager.cc:1347] Generated new cluster ID: 3579eb9c3ddf440fb4d4d3a23dded176
I20260812 06:57:05.752557 26479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 06:57:05.752588 26503 mvcc.cc:205] Tried to move back new op lower bound from 7317577014079565824 to 7317577013759864832. Current Snapshot: MvccSnapshot[applied={T|T < 7317577014079565824 or (T in {7317577014079565824})}]
I20260812 06:57:05.755877 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.756179 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:05.757252 26286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.757614 26286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a90d56f8c7434d9e740faf8dde9998, State: Initialized, Role: FOLLOWER
I20260812 06:57:05.758065 26286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } }
I20260812 06:57:05.767836 26286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:57:05.768294 26233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:57:05.771938 26233 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [term 1 FOLLOWER]: Refusing update from remote peer ab36b3105cc5497191fe07f9fbdb87d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:57:05.772850 26475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:57:05.772934 26508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.773280 26508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.773389 26236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "63a90d56f8c7434d9e740faf8dde9998"
I20260812 06:57:05.776671 26236 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab36b3105cc5497191fe07f9fbdb87d5 in term 1.
I20260812 06:57:05.786132 26479 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 06:57:05.787868 26479 catalog_manager.cc:1504] Loading token signing keys...
I20260812 06:57:05.800225 26479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5: Generated new TSK 0
I20260812 06:57:05.800925 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.801235 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab36b3105cc5497191fe07f9fbdb87d5 [sys.catalog]: This master's current role is: LEADER
I20260812 06:57:05.801733 26479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 06:57:05.802767 26508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab36b3105cc5497191fe07f9fbdb87d5. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.803184 26508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.806463 26526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab36b3105cc5497191fe07f9fbdb87d5" member_type: VOTER last_known_addr { host: "127.17.69.126" port: 37733 } } peers { permanent_uuid: "63a90d56f8c7434d9e740faf8dde9998" member_type: VOTER last_known_addr { host: "127.17.69.125" port: 41447 } } peers { permanent_uuid: "0473628c45d04868a28980b28609078a" member_type: VOTER last_known_addr { host: "127.17.69.124" port: 39553 } } }
I20260812 06:57:05.806864 26526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:57:05.808550 26530 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260812 06:57:05.901015 26142 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.892713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.17.69.126:37733)
0812 06:57:05.893042 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.893048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.898892 (+ 5844us) server_negotiation.cc:407] Connection header received
0812 06:57:05.899204 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.899212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.899332 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.899411 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.900732 (+ 1321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36126: BlockingRecv error: recv got EOF from 127.0.0.1:36126 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
W20260812 06:57:05.906323 26494 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.892680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.17.69.124:39553)
0812 06:57:05.892883 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.892889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.904323 (+ 11434us) server_negotiation.cc:407] Connection header received
0812 06:57:05.904499 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.904508 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.904617 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.904719 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.906073 (+ 1354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38326: BlockingRecv error: recv got EOF from 127.0.0.1:38326 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 06:57:05.906666 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.899003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.17.69.125:41447)
0812 06:57:05.899182 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.899189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.905033 (+ 5844us) server_negotiation.cc:407] Connection header received
0812 06:57:05.905264 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.905270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.905364 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.905432 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.906444 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57610: BlockingRecv error: recv got EOF from 127.0.0.1:57610 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 06:57:05.915962 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.913762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.17.69.125:41447)
0812 06:57:05.913946 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.913951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.914468 (+ 517us) server_negotiation.cc:407] Connection header received
0812 06:57:05.914690 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.914696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.914819 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.914930 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.915734 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57616: BlockingRecv error: recv got EOF from 127.0.0.1:57616 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 06:57:05.924216 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.920357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.17.69.125:41447)
0812 06:57:05.920510 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.920515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.922835 (+ 2320us) server_negotiation.cc:407] Connection header received
0812 06:57:05.922996 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.923002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.923083 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.923201 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.924012 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57618: BlockingRecv error: recv got EOF from 127.0.0.1:57618 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 06:57:05.938282 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.936272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.17.69.125:41447)
0812 06:57:05.936419 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.936425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.936879 (+ 454us) server_negotiation.cc:407] Connection header received
0812 06:57:05.937085 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.937090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.937187 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.937296 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.938049 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57628: BlockingRecv error: recv got EOF from 127.0.0.1:57628 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 06:57:05.976649 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.974585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.17.69.125:41447)
0812 06:57:05.974788 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.974794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.975336 (+ 542us) server_negotiation.cc:407] Connection header received
0812 06:57:05.975514 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.975519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.975607 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.975706 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.976441 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57634: BlockingRecv error: recv got EOF from 127.0.0.1:57634 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 06:57:05.989880 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:05.987927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.17.69.125:41447)
0812 06:57:05.988075 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:05.988080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:05.988586 (+ 506us) server_negotiation.cc:407] Connection header received
0812 06:57:05.988760 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:05.988765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:05.988856 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:05.988955 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:05.989688 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57650: BlockingRecv error: recv got EOF from 127.0.0.1:57650 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 06:57:06.010993 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.008922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.17.69.125:41447)
0812 06:57:06.009084 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.009090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.009583 (+ 493us) server_negotiation.cc:407] Connection header received
0812 06:57:06.009756 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.009761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.009853 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.009947 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.010792 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57652: BlockingRecv error: recv got EOF from 127.0.0.1:57652 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 06:57:06.048586 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.046590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.17.69.125:41447)
0812 06:57:06.046794 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.046801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.047144 (+ 343us) server_negotiation.cc:407] Connection header received
0812 06:57:06.047358 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.047362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.047435 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.047519 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.048329 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57656: BlockingRecv error: recv got EOF from 127.0.0.1:57656 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 06:57:06.118255 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.116255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.17.69.125:41447)
0812 06:57:06.116489 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.116495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.116878 (+ 383us) server_negotiation.cc:407] Connection header received
0812 06:57:06.117089 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.117094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.117180 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.117267 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.118045 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57664: BlockingRecv error: recv got EOF from 127.0.0.1:57664 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 06:57:06.251439 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.249603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.17.69.125:41447)
0812 06:57:06.249770 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.249775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.250234 (+ 459us) server_negotiation.cc:407] Connection header received
0812 06:57:06.250409 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.250413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.250488 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.250572 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.251197 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57668: BlockingRecv error: recv got EOF from 127.0.0.1:57668 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 06:57:06.512437 26292 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 06:57:06.510646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.17.69.125:41447)
0812 06:57:06.510878 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:06.510884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:06.511101 (+ 217us) server_negotiation.cc:407] Connection header received
0812 06:57:06.511321 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:06.511326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:06.511400 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:06.511485 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 06:57:06.512236 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57682: BlockingRecv error: recv got EOF from 127.0.0.1:57682 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 06:57:06.534538 26454 catalog_manager.cc:1259] Loaded cluster ID: 3579eb9c3ddf440fb4d4d3a23dded176
I20260812 06:57:06.534615 26454 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: loading cluster ID for follower catalog manager: success
I20260812 06:57:06.536945 26454 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: acquiring CA information for follower catalog manager: success
I20260812 06:57:06.538645 26454 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0473628c45d04868a28980b28609078a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:57:06.812652 26530 catalog_manager.cc:1259] Loaded cluster ID: 3579eb9c3ddf440fb4d4d3a23dded176
I20260812 06:57:06.812824 26530 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: loading cluster ID for follower catalog manager: success
I20260812 06:57:06.815515 26530 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: acquiring CA information for follower catalog manager: success
I20260812 06:57:06.817294 26530 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63a90d56f8c7434d9e740faf8dde9998: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:57:07.033843 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.027829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.17.69.125:41447)
0812 06:57:07.028589 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.028594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.028609 (+ 15us) server_negotiation.cc:407] Connection header received
0812 06:57:07.028671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.028675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.028807 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.028915 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:07.029373 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.030225 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.031553 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.032021 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.032138 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.032633 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:07.033189 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:07.033323 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:07.033440 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 06:57:07.033517 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":448,"threads_started":1}
I20260812 06:57:07.045917 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.039366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.17.69.124:39553)
0812 06:57:07.039916 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.039922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.040264 (+ 342us) server_negotiation.cc:407] Connection header received
0812 06:57:07.040476 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.040481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.040620 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.040768 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:07.041219 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.041905 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.043811 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.044194 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.044284 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.044879 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:07.045330 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:07.045431 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:07.045564 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 06:57:07.045627 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
I20260812 06:57:07.066308 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.052722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.17.69.124:39553)
0812 06:57:07.052904 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.052910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.059375 (+ 6465us) server_negotiation.cc:407] Connection header received
0812 06:57:07.059778 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.059783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.059876 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.059940 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:07.060687 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.061795 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.064092 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.064745 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.064835 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.065491 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:07.065628 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:07.065687 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:07.065899 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 06:57:07.065981 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 06:57:07.067618 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.052200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.17.69.126:37733)
0812 06:57:07.059018 (+ 6818us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.059024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.059041 (+ 17us) server_negotiation.cc:407] Connection header received
0812 06:57:07.059149 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.059155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.059363 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.059538 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:07.060282 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.061370 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.063578 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.064217 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.064331 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.064997 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:07.065411 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:07.065522 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:07.067267 (+ 1745us) server_negotiation.cc:299] Negotiation successful
0812 06:57:07.067347 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6695,"thread_start_us":6578,"threads_started":1}
I20260812 06:57:07.072175 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 06:57:07.059486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.17.69.125:41447)
0812 06:57:07.059643 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 06:57:07.059649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 06:57:07.066081 (+ 6432us) server_negotiation.cc:407] Connection header received
0812 06:57:07.066363 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 06:57:07.066370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 06:57:07.066460 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 06:57:07.066527 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 06:57:07.067304 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.068379 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.070138 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 06:57:07.070821 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 06:57:07.070992 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 06:57:07.071449 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 06:57:07.071573 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 06:57:07.071684 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 06:57:07.071862 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 06:57:07.071932 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 06:57:07.074447 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26025
I20260812 06:57:07.075464 26132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:07.181380 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26025
W20260812 06:57:07.206234 26190 connection.cc:535] server connection from 127.0.0.1:56460 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:57:07.206877 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26148
I20260812 06:57:07.207873 26275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:07.310712 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26148
I20260812 06:57:07.337407 17685 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu pid 26298
I20260812 06:57:07.338567 26411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 06:57:07.441951 17685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknIa6XT/build/asan/bin/kudu with pid 26298
2026-08-12T06:57:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4943 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9647 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (361195 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST